Erich Kummerfeld

Erratum to: Synthese DOI 10.1007/s11229-014-0408-3Appendix 1: NotationLet \(X\) represent a sequence of data, and let \(X_B^t\) represent an i.i.d. subsequence of length \(t\) of data generated from distribution \(B\).We conjecture that the i.i.d. assumption could be eliminated by defining probability distributions over sequences of arbitrary length, though this complication would not add conceptual clarity. Let \(\mathbf{F}\) be a framework (in this case, a set of probability distributions or densities).Let any \(P(\,)\) functions be either a probability distribution function or probability density function, as appropriate. Let \(M_\mathbf{F}\) be a method that takes a data sequence \(X\) as input and outputs a distribution

One persistent challenge in scientific practice is that the structure of the world can be unstable: changes in the broader context can alter which model of a phenomenon is preferred, all without any overt signal. Scientific discovery becomes much harder when we have a moving target, and the resulting incorrect understandings of relationships in the world can have significant real-world and practical consequences. In this paper, we argue that it is common (in certain sciences) to have changes of context that lead to changes in the relationships under study, but that standard normative accounts of scientific inquiry have assumed away this problem. At the same time, we show that inference and discovery methods can “protect” themselves in various ways against this possibility by using methods with the novel methodological virtue of “diligence.” Unfortunately, this desirable virtue provably is incompatible with other desirable methodological virtues that are central to reliable inquiry. No scientific method can provide every virtue that we might want

Current fears of a “reproducibility crisis” have led researchers, sources of scientific funding, and the public to question both the efficacy and trustworthiness of science. Suggested policy changes have been focused on statistical problems, such as p-hacking, and issues of experimental design and execution. However, “reproducibility” is a broad concept that includes a number of issues. Furthermore, reproducibility failures occur even in fields such as mathematics or computer science that do not have statistical problems or issues with experimental design. Most importantly, these proposed policy changes ignore a core feature of the process of scientific inquiry that occurs after reproducibility failures: the integration of conflicting observations and ideas into a coherent theory.

Those who comment on modern scientific institutions are often quick to praise institutional structures that leave scientists to their own devices. These comments reveal an underlying presumption that scientists do best when left alone—when they operate in what we call the ‘scientific state of nature’. Through computer simulation, we challenge this presumption by illustrating an inefficiency that arises in the scientific state of nature. This inefficiency suggests that one cannot simply presume that science is most efficient when institutional control is absent. In some situations, actively encouraging unpopular, risky science would improve scientific outcomes. 1 Introduction2 Scientists and Bandits3 Choosing an ϵ4 Structure of Communication5 Discussion