•  170
    Epistemology of ignorance: the contribution of philosophy to the science-policy interface of marine biosecurity
    with Anne Schwenkenbecher, Chad L. Hewitt, Marnie L. Campbell, Oliver Fritsch, Andrew T. Knight, and Erin Nash
    Frontiers in Marine Science 10 1-5. 2023.
    Marine ecosystems are under increasing pressure from human activity, yet successful management relies on knowledge. The evidence-based policy (EBP) approach has been promoted on the grounds that it provides greater transparency and consistency by relying on ‘high quality’ information. However, EBP also creates epistemic responsibilities. Decision-making where limited or no empirical evidence exists, such as is often the case in marine systems, creates epistemic obligations for new information ac…Read more
  •  229
    Perspectives on Scientific Error
    with Don van Ravenzwaaij, Marjan Bakker, Felipe Romero, Noah van Dongen, Sophia Crüwell, Sarahanne Field, Leonard Held, Marcus Munafò, Merle-Marie Pittelkow, Leonid Tiokhin, Vincent Traag, Olmo van den Akker, Anna van 'T. Veer, and Eric Jan Wagenmakers
    Royal Society Open Science 10 (7): 230448. 2023.
    Theoretical arguments and empirical investigations indicate that a high proportion of published findings do not replicate and are likely false. The current position paper provides a broad perspective on scientific error, which may lead to replication failures. This broad perspective focuses on reform history and on opportunities for future reform. We organize our perspective along four main themes: institutional reform, methodological reform, statistical reform and publishing reform. For each th…Read more
  •  136
    This note contains a corrective and a generalization of results by Borsboom et al. (2008), based on Heesen and Romeijn (2019). It highlights the relevance of insights from psychometrics beyond the context of psychological testing.
  •  326
    To Be Scientific Is To Be Communist
    Social Epistemology 37 (3): 249-258. 2023.
    What differentiates scientific research from non-scientific inquiry? Philosophers addressing this question have typically been inspired by the exalted social place and intellectual achievements of science. They have hence tended to point to some epistemic virtue or methodological feature of science that sets it apart. Our discussion on the other hand is motivated by the case of commercial research, which we argue is distinct from (and often epistemically inferior to) academic research. We consid…Read more
  •  57
    Cumulative Advantage and the Incentive to Commit Fraud in Science
    British Journal for the Philosophy of Science. forthcoming.
    This paper investigates how the credit incentive to engage in questionable research practices interacts with cumulative advantage, the process whereby high-status academics more easily increase their status than low-status academics. I use a mathematical model to highlight two dynamics that have not yet received much attention. First, due to cumulative advantage, questionable research practices may pay off over the course of an academic career even if they are not attractive at the level of indi…Read more
  •  27
    This paper investigates how the credit incentive to engage in questionable research practices interacts with cumulative advantage, the process whereby high-status academics more easily increase their status than low-status academics. I use a mathematical model to highlight two dynamics that have not yet received much attention. First, due to cumulative advantage, questionable research practices may pay off over the course of an academic career even if they do not appear attractive at the level o…Read more
  •  263
    This paper offers a new angle on the common idea that the process of science does not support epistemic diversity. Under minimal assumptions on the nature of journal editing, we prove that editorial procedures, even when impartial in themselves, disadvantage less prominent research programs. This purely statistical bias in article selection further skews existing differences in the success rate and hence attractiveness of research programs, and exacerbates the reputation difference between the p…Read more
  •  409
    Jury Theorems for Peer Review
    British Journal for the Philosophy of Science. forthcoming.
    Peer review is often taken to be the main form of quality control on academic research. Usually journals carry this out. However, parts of maths and physics appear to have a parallel, crowd-sourced model of peer review, where papers are posted on the arXiv to be publicly discussed. In this paper we argue that crowd-sourced peer review is likely to do better than journal-solicited peer review at sorting papers by quality. Our argument rests on two key claims. First, crowd-sourced peer review will…Read more
  •  239
    The Necessity of Commensuration Bias in Grant Peer Review
    Ergo: An Open Access Journal of Philosophy 8 (39): 423--443. 2021.
    Peer reviewers at many funding agencies and scientific journals are asked to score submissions both on individual criteria and overall. The overall scores should be some kind of aggregate of the criteria scores. Carole Lee identifies this as a potential locus for bias to enter the peer review process, which she calls commensuration bias. Here I view the aggregation of scores through the lens of social choice theory. I argue that, when reviewing grant proposals, it is in many cases impossible to …Read more
  •  31
    Review of the volume "Scientific Collaboration and Collective Knowledge: New Essays", edited by Thomas Boyer-Kassem, Conor Mayo-Wilson, and Michael Weisberg.
  •  2174
    Why the Reward Structure of Science Makes Reproducibility Problems Inevitable
    Journal of Philosophy 115 (12): 661-674. 2018.
    Recent philosophical work has praised the reward structure of science, while recent empirical work has shown that many scientific results may not be reproducible. I argue that the reward structure of science incentivizes scientists to focus on speed and impact at the expense of the reproducibility of their work, thus contributing to the so-called reproducibility crisis. I use a rational choice model to identify a set of sufficient conditions for this problem to arise, and I argue that these cond…Read more
  •  36
    Recent empirical work has shown that many scientific results may not be reproducible. By itself, this does not entail that there is a problem. However, I argue that there is a problem: the reward structure of science incentivizes scientists to focus on speed and impact at the expense of the reproducibility of their work. I illustrate this using a well-known failure of reproducibility: Fleischmann and Pons' work on cold fusion. I then use a rational choice model to identify a set of sufficient co…Read more
  •  306
    The credit incentive to be a maverick
    Studies in History and Philosophy of Science Part A 76 5-12. 2019.
    There is a commonly made distinction between two types of scientists: risk-taking, trailblazing mavericks and detail-oriented followers. A number of recent papers have discussed the question what a desirable mixture of mavericks and followers looks like. Answering this question is most useful if a scientific community can be steered toward such a desirable mixture. One attractive route is through credit incentives: manipulating rewards so that reward-seeking scientists are likely to form the des…Read more
  •  1293
    Is Peer Review a Good Idea?
    British Journal for the Philosophy of Science 72 (3): 635-663. 2021.
    Prepublication peer review should be abolished. We consider the effects that such a change will have on the social structure of science, paying particular attention to the changed incentive structure and the likely effects on the behaviour of individual scientists. We evaluate these changes from the perspective of epistemic consequentialism. We find that where the effects of abolishing prepublication peer review can be evaluated with a reasonable level of confidence based on presently available …Read more
  •  27
    I discuss a game-theoretic model in which scientists compete to finish the intermediate stages of some research project. Banerjee et al. have previously shown that if the credit awarded for intermediate results is proportional to their difficulty, then the strategy profile in which scientists share each intermediate stage as soon as they complete it is a Nash equilibrium. I show that the equilibrium is both unique and strict. Thus rational credit-maximizing scientists have an incentive to share …Read more
  •  532
    Vindicating methodological triangulation
    with Liam Kofi Bright and Andrew Zucker
    Synthese 196 (8): 3067-3081. 2019.
    Social scientists use many different methods, and there are often substantial disagreements about which method is appropriate for a given research question. In response to this uncertainty about the relative merits of different methods, W. E. B. Du Bois advocated for and applied “methodological triangulation”. This is to use multiple methods simultaneously in the belief that, where one is uncertain about the reliability of any given method, if multiple methods yield the same answer that answer i…Read more
  •  480
    This paper addresses the problem of judgment aggregation in science. How should scientists decide which propositions to assert in a collaborative document? We distinguish the question of what to write in a collaborative document from the question of collective belief. We argue that recent objections to the application of the formal literature on judgment aggregation to the problem of judgment aggregation in science apply to the latter, not the former question. The formal literature has introduce…Read more
  •  70
    The Bounded Strength of Weak Expectations
    Mind 120 (479): 819-832. 2011.
    The rational price of the Pasadena and Altadena games, introduced by Nover and Hájek (2004 ), has been the subject of considerable discussion. Easwaran (2008 ) has suggested that weak expectations — the value to which the average payoffs converge in probability — can give the rational price of such games. We argue against the normative force of weak expectations in the standard framework. Furthermore, we propose to replace this framework by a bounded utility perspective: this shift renders the p…Read more
  •  433
    Communism and the Incentive to Share in Science
    Philosophy of Science 84 (4): 698-716. 2017.
    The communist norm requires that scientists widely share the results of their work. Where did this norm come from, and how does it persist? Michael Strevens provides a partial answer to these questions by showing that scientists should be willing to sign a social contract that mandates sharing. However, he also argues that it is not in an individual credit-maximizing scientist's interest to follow this norm. I argue against Strevens that individual scientists can rationally conform to the commun…Read more
  •  360
    When journal editors play favorites
    Philosophical Studies 175 (4): 831-858. 2018.
    Should editors of scientific journals practice triple-anonymous reviewing? I consider two arguments in favor. The first says that insofar as editors’ decisions are affected by information they would not have had under triple-anonymous review, an injustice is committed against certain authors. I show that even well-meaning editors would commit this wrong and I endorse this argument. The second argument says that insofar as editors’ decisions are affected by information they would not have had und…Read more
  •  446
    How much evidence should one collect?
    Philosophical Studies 172 (9): 2299-2313. 2015.
    A number of philosophers of science and statisticians have attempted to justify conclusions drawn from a finite sequence of evidence by appealing to results about what happens if the length of that sequence tends to infinity. If their justifications are to be successful, they need to rely on the finite sequence being either indefinitely increasing or of a large size. These assumptions are often not met in practice. This paper analyzes a simple model of collecting evidence and finds that the prac…Read more
  •  420
    A Game-Theoretic Approach to Peer Disagreement
    Erkenntnis 81 (6): 1345-1368. 2016.
    In this paper we propose and analyze a game-theoretic model of the epistemology of peer disagreement. In this model, the peers' rationality is evaluated in terms of their probability of ending the disagreement with a true belief. We find that different strategies---in particular, one based on the Steadfast View and one based on the Conciliatory View---are rational depending on the truth-sensitivity of the individuals involved in the disagreement. Interestingly, the Steadfast and the Conciliatory…Read more
  •  38
    It is well-known that some scientists are more prominent than others. But what makes one scientist more prominent than another? I propose a possible mechanism that produces differences in prominence: scientists' desire for information. In a model of a scientific community exchanging information, I show that this mechanism indeed produces the kind of patterns of prominence that are actually observed. I discuss the implications of this result for three possible explanations of an individual scient…Read more
  •  419
    Academic superstars: competent or lucky?
    Synthese 194 (11): 4499-4518. 2017.
    I show that the social stratification of academic science can arise as a result of academics’ preference for reading work of high epistemic value. This is consistent with a view on which academic superstars are highly competent academics, but also with a view on which superstars arise primarily due to luck. I argue that stratification is beneficial if most superstars are competent, but not if most superstars are lucky. I also argue that it is impossible to tell whether most superstars are in fac…Read more
  •  420
    In the second volume of the "Handbuch der physiologischen Optik", published in 1860, Helmholtz sets out a three-receptor theory of color vision using coterminal response curves, and shows that this theory can unify most phenomena of color mixing known at the time. Maxwell had publicized the same theory five years earlier, but Helmholtz barely acknowledges this fact in the "Handbuch". Some historians have argued that this is because Helmholtz independently discovered the theory around the same ti…Read more