-
11This dissertation is a study of how methodological issues in psychology can have significant implications for philosophical accounts of interpretation, justification, and psychological explanation. In the first chapter, I analyze traditional philosophical accounts of interpretation with an eye to identifying the ways in which philosophers have used rationality as a methodological tool. I argue that these forms of methodological rationalism do not successfully cope with the challenge from the heu…Read more
-
7NIH Peer Review: Criterion Scores Completely Account for Racial Disparities in Overall Impact ScoresScience Advances 6 (23). 2020.Previous research has found that funding disparities are driven by applications’ final impact scores and that only a portion of the black/white funding gap can be explained by bibliometrics and topic choice. Using National Institutes of Health R01 applications for council years 2014–2016, we examine assigned reviewers’ preliminary overall impact and criterion scores to evaluate whether racial disparities in impact scores can be explained by application and applicant characteristics. We hypothesi…Read more
-
12When Zero May Not be Zero: A Cautionary Note on the Use of Inter-Rater Reliability in Evaluating Grant Peer ReviewJournal of the Royal Statistical Society: Series A 184 904-19. 2021.Considerable attention has focused on studying reviewer agreement via inter-rater reliability (IRR) as a way to assess the quality of the peer review process. Inspired by a recent study that reported an IRR of zero in the mock peer review of top-quality grant proposals, we use real data from a complete range of submissions to the National Institutes of Health and to the American Institute of Biological Sciences to bring awareness to two important issues with using IRR for assessing peer review q…Read more
-
Refinement: Measuring informativeness of ratings in the absence of a gold standardBritish Journal of Mathematical and Statistical Psychology 75 (3): 593-615. 2022.We propose a new metric for evaluating the informativeness of a set of ratings from a single rater on a given scale. Such evaluations are of interest when raters rate numerous comparable items on the same scale, as occurs in hiring, college admissions, and peer review. Our exposition takes the context of peer review, which involves univariate and multivariate cardinal ratings. We draw on this context to motivate an information-theoretic measure of the refinement of a set of ratings – entropic re…Read more
-
32Certified Amplification: An Emerging Scientific Norm and EthosPhilosophy of Science 89 (5): 1002-1012. 2022.Merton envisioned his norms of science at a time when peer-reviewed journals controlled scientific communication. Technologies for sharing and finding content have since divorced the certification and amplification of science, generating systemic vulnerabilities. Certified amplification—a new Mertonian-styled norm—enjoins their recoupling and introduces a taxonomy of strategies adopted by institutions to close the certification-amplification gap, including the proportioning of the one to the oth…Read more
-
1Alternative Funding Models Might Perpetuate Black-White Funding GapsThe Lancet 396 955-6. 2020.The White Coats for Black Lives and #ShutDownSTEM movements have galvanised biomedical practitioners and researchers to eliminate institutional and systematic racism, including barriers faced by Black researchers in biomedicine and science, technology, engineering, and mathematics. In our study on Black–White funding gaps for National Institutes of Health Research Project grants, we found that the overall award rate for Black applicants is 55% of that for white applicants. How can systems for al…Read more
-
100The Reference Class Problem for Credit Valuation in SciencePhilosophy of Science 87 (5): 1026-1036. 2020.Scholars belong to multiple communities of credit simultaneously. When these communities disagree about a scholarly achievement’s credit assignment, this raises a puzzle for decision and game theor...
-
48Gender-based homophily in collaborations across a heterogeneous scholarly landscapePLoS ONE 18 (4). 2023.Using the corpus of JSTOR articles, we investigate the role of gender in collaboration patterns across the scholarly landscape by analyzing gender-based homophily--the tendency for researchers to co-author with individuals of the same gender. For a nuanced analysis of gender homophily, we develop methodology necessitated by the fact that the data comprises heterogeneous sub-disciplines and that not all authorships are exchangeable. In particular, we distinguish three components of gender homophi…Read more
-
21Promoting Gender Equity in Grant Making: What Can a Funder Do?The Lancet 393 (10171). 2019.The Doris Duke Charitable Foundation's competitive career development award selects awardees annually. This paper describes changes DDCF made to its grants making process to improve gender representation in its applicant and awardee pools.
-
28Promote Scientific Integrity via Journal Peer Review DataScience 357 (6348): 256-257. 2017.There is an increasing push by journals to ensure that data and products related to published papers are shared as part of a cultural move to promote transparency, reproducibility, and trust in the scientific literature. Yet few journals commit to evaluating their effectiveness in implementing reporting standards aimed at meeting those goals (1, 2). Similarly, though the vast majority of journals endorse peer review as an approach to ensure trust in the literature, few make their peer review dat…Read more
-
348A Dispositional Account of Aversive RacismProceedings of the 40th Annual Cognitive Science Society. 2018.I motivate and articulate a dispositional account of aversive racism. By conceptualizing and measuring attitudes in terms of their full distribution, rather than in terms of their mode or mean preference, my account of dispositional attitudes gives ambivalent attitudes (qua attitude) the ability to predict aggregate behavior. This account can be distinguished from other dispositional accounts of attitude by its ability to characterize ambivalent attitudes such as aversive racism at the attitudin…Read more
-
457Collective Implicit Attitudes: A Stakeholder Conception of Implicit BiasProceedings of the 40th Annual Cognitive Science Society. 2018.Psychologists and philosophers have not yet resolved what they take implicit attitudes to be; and, some, concerned about limitations in the psychometric evidence, have even challenged the predictive and theoretical value of positing implicit attitudes in explanations for social behavior. In the midst of this debate, prominent stakeholders in science have called for scientific communities to recognize and countenance implicit bias in STEM fields. In this paper, I stake out a stakeholder conceptio…Read more
-
108Revisiting Current Causes of Women's Underrepresentation in ScienceIn Michael Brownstein & Jennifer Mather Saul (eds.), Implicit Bias and Philosophy, Volume 1: Metaphysics and Epistemology, Oxford University Press. 2016.On the surface, developing a social psychology of science seems compelling as a way to understand how individual social cognition – in aggregate – contributes towards individual and group behavior within scientific communities (Kitcher, 2002). However, in cases where the functional input-output profile of psychological processes cannot be mapped directly onto the observed behavior of working scientists, it becomes clear that the relationship between psychological claims and normative philosophy…Read more
-
74Gricean charity: The Gricean turn in psychologyPhilosophy of the Social Sciences 36 (2): 193-218. 2006.Psychologists' work on conversational pragmatics and judgment suggests a refreshing approach to charitable interpretation and theorizing. This charitable approach—what I call Gricean charity —recognizes the role of conversational assumptions and norms in subject-experimenter communication. In this paper, I outline the methodological lessons Gricean charity gleans from psychologists' work in conversational pragmatics. In particular, Gricean charity imposes specific evidential standards requiring …Read more
-
162Bias in Peer ReviewJournal of the American Society for Information Science and Technology 64 (1): 2-17. 2013.Research on bias in peer review examines scholarly communication and funding processes to assess the epistemic and social legitimacy of the mechanisms by which knowledge communities vet and self-regulate their work. Despite vocal concerns, a closer look at the empirical and methodological limitations of research on bias raises questions about the existence and extent of many hypothesized forms of bias. In addition, the notion of bias is predicated on an implicit ideal that, once articulated, rai…Read more
-
142Social Biases and Solution for Procedural ObjectivityHypatia 26 (2): 352-73. 2011.An empirically sensitive formulation of the norms of transformative criticism must recognize that even public and shared standards of evaluation can be implemented in ways that unintentionally perpetuate and reproduce forms of social bias that are epistemically detrimental. Helen Longino’s theory can explain and redress such social bias by treating peer evaluations as hypotheses based on data and by requiring a kind of perspectival diversity that bears, not on the content of the community’s know…Read more
-
607Reclaiming Davidson’s Methodological Rationalism as Galilean Idealization in PsychologyPhilosophy of the Social Sciences 40 (1): 84-106. 2010.In his early experimental work with Suppes, Davidson adopted rationality assumptions, not as necessary constraints on interpretation, but as practical conceits in addressing methodological problems faced by experimenters studying decision making under uncertainty. Although the content of their theory has since been undermined, their methodological approach—a Galilean form of methodological rationalism—lives on in contemporary psychological research. This article draws on Max Weber’s verstehen to…Read more
-
106Commensuration Bias in Peer ReviewPhilosophy of Science 82 (5). 2015.To arrive at their final evaluation of a manuscript or grant proposal, reviewers must convert a submission’s strengths and weaknesses for heterogeneous peer review criteria into a single metric of quality or merit. I identify this process of commensuration as the locus for a new kind of peer review bias. Commensuration bias illuminates how the systematic prioritization of some peer review criteria over others permits and facilitates problematic patterns of publication and funding in science. Com…Read more
-
635The limited effectiveness of prestige as an intervention on the health of medical journal publicationsEpisteme 10 (4): 387-402. 2013.Under the traditional system of peer-reviewed publication, the degree of prestige conferred to authors by successful publication is tied to the degree of the intellectual rigor of its peer review process: ambitious scientists do well professionally by doing well epistemically. As a result, we should expect journal editors, in their dual role as epistemic evaluators and prestige-allocators, to have the power to motivate improved author behavior through the tightening of publication requirements. …Read more
-
31Asian Americans, positive stereotyping, and philosophyAmerican Philosophical Association Newsletter on Asian and Asian-American Philosophers and Philosophies 14 (2-7). 2014.What is the current status of Asian Americans in philosophy? How do Asian Americans fare in comparison to other minority groups? And, what professional strategies might they use (more or less successfully) in response to their counterstereotypical status in philosophy? In this piece, I will address these questions empirically by extrapolating from available demographic, survey, and experimental studies. This analysis will be too fast and loose, but I offer it in the spirit of constructing a broa…Read more
-
81The representation of judgment heuristics and the generality problemProceedings of the 29th Annual Cognitive Science Society 1211-6. 2007.In his debates with Daniel Kahneman and Amos Tversky, Gerd Gigerenzer puts forward a stricter standard for the proper representation of judgment heuristics. I argue that Gigerenzer’s stricter standard contributes to naturalized epistemology in two ways. First, Gigerenzer’s standard can be used to winnow away cognitive processes that are inappropriately characterized and should not be used in the epistemic evaluation of belief. Second, Gigerenzer’s critique helps to recast the generality problem …Read more
-
52Philosophy journal practices and opportunities for biasAmerican Philosophical Association Newsletter on Feminism and Philosophy. 2010.
-
101Applied cognitive psychology and the "strong replacement" of epistemology by normative psychologyPhilosophy of the Social Sciences 38 (1): 55-75. 2008.is normative in the sense that it aims to make recommendations for improving human judgment; it aims to have a practical impact on morally and politically significant human decisions and actions; and it studies normative, rational judgment qua rational judgment. These nonstandard ways of understanding ACP as normative collectively suggest a new interpretation of the strong replacement thesis that does not call for replacing normative epistemic concepts, relations, and inquiries with descriptive,…Read more
-
90A Kuhnian Critique of Psychometric Research on Peer ReviewPhilosophy of Science 79 (5): 859-870. 2012.Psychometrically oriented researchers construe low inter-rater reliability measures for expert peer reviewers as damning for the practice of peer review. I argue that this perspective overlooks different forms of normatively appropriate disagreement among reviewers. Of special interest are Kuhnian questions about the extent to which variance in reviewer ratings can be accounted for by normatively appropriate disagreements about how to interpret and apply evaluative criteria within disciplines …Read more
APA Western Division
Seattle, Washington, United States of America
Areas of Specialization
Philosophy of Cognitive Science |
Philosophy of Social Science |