Kent Staley

: For philosophers of science interested in elucidating the social character of science, an important question concerns the manner in which and degree to which the objectivity of scientific knowledge is socially constituted. We address this broad question by focusing specifically on philosophical theories of evidence. To get at the social character of evidence, we take an interdisciplinary approach informed by categories from argumentation studies. We then test these categories by exploring their applicability to a case study from high-energy physics. Our central claim is that normative philosophy of science must move beyond abstract theories of justification, confirmation, or evidence conceived impersonally and incorporate a theoretical perspective that includes dialogical elements, either as adjuncts to impersonal theories of evidence or as intrinsic to the cogency of scientific argumentation.

Reports of quantitative experimental results often distinguish between the statistical uncertainty and the systematic uncertainty that characterize measurement outcomes. This article discusses the practice of estimating systematic uncertainty in high-energy physics. The estimation of systematic uncertainty in HEP should be understood as a minimal form of quantitative robustness analysis. The secure evidence framework is used to explain the epistemic significance of robustness analysis. However, the empirical value of a measurement result depends crucially not only on the resulting systematic uncertainty estimate, but on the learning aims for which that result will be used. Important conceptual and practical questions regarding systematic uncertainty assessment call for further investigation. _1_ Introduction _2_ Systematic Uncertainty: The Very Idea _3_ Systematic Uncertainty in High-Energy Physics _4_ Methodological Debates in High-Energy Physics _5_ The Secure Evidence Framework _6_ Systematic Uncertainty Assessment as Robustness Analysis _7_ Security and Sensitivity _8_ Conclusion Appendix

This study presents a historical account of experiments, performed by the Collider Detector at Fermilab collaboration, which led to the discovery of the top quark, and a discussion of philosophical issues raised by that episode. ;The historical discussion is based on published and unpublished documents and oral history interviews, and is presented in two parts: First, the formation of the collaboration and construction of the detector are described. The activities of the collaborators during the period of detector construction are described in terms of the development of resources for a general experimental programme. Second, the development of the means of analyzing the data for the top quark search is described, particularly aspects of the analysis that were disputed. The hypothesis that collaboration researchers have come to regard the social process of resolving disputes as a matter of methodological importance is suggested. ;The philosophical discussion of the experiment employs the hierarchy of models approach of Patrick Suppes and Deborah Mayo in order to examine the logic of hypothesis testing and draw some conclusions regarding the nature of scientific evidence. In an extension of an argument presented by Peter Achinstein, the account of hypothesis testing given by hypothetico-deductivist philosophers such as Karl Popper and R. B. Braithwaite is examined in light of the reasoning employed in the top search, and is found wanting. The prediction based on the hypothesis being tested in the top search is found to have been inferred inductively from the experimental data. ;Finally, a discussion is presented of tuning on the signal, a form of bias in the testing of hypotheses. The proscription of this form of bias resembles John Worrall's requirement of use novelty, but is shown instead to serve the aim of devising a test of the hypothesis that is severe, in the sense articulated by Deborah Mayo. It is shown that the evaluation of evidence claims, as it depends on severity considerations, sometimes, as in the case of the top search, requires consideration of the procedures used in designing the test, which may involve historical and social facts about the experimenters and their actions

Experimenters sometimes insist that it is unwise to examine data before determining how to analyze them, as it creates the potential for biased results. I explore the rationale behind this methodological guideline from the standpoint of an error statistical theory of evidence, and I discuss a method of evaluating evidence in some contexts when this predesignation rule has been violated. I illustrate the problem of potential bias, and the method by which it may be addressed, with an example from the search for the top quark. A point in favor of the error statistical theory is its ability, demonstrated here, to explicate such methodological problems and suggest solutions, within the framework of an objective theory of evidence.

Some accounts of evidence regard it as an objective relationship holding between data and hypotheses, perhaps mediated by a testing procedure. Mayo’s error-statistical theory of evidence is an example of such an approach. Such a view leaves open the question of when an epistemic agent is justified in drawing an inference from such data to a hypothesis. Using Mayo’s account as an illustration, I propose a framework for addressing the justification question via a relativized notion, which I designate security , meant to conceptualize practices aimed at the justification of inferences from evidence. I then show how the notion of security can be put to use by showing how two quite different theoretical approaches to model criticism in statistics can both be viewed as strategies for securing claims about statistical evidence

It is sometimes held that facts confirm a hypothesis only if they were not used in the construction of that hypothesis. This requirement of "use novelty" introduces a historical aspect into the assessment of evidence claims. I examine a methodological principle invoked by physicists in the experimental search for the top quark that bears a striking resemblance to this view. However, this principle is better understood, both historically and philosophically, in terms of the need to conduct a severe test than in terms of use novelty. Nevertheless, a historical factor remains in the assessment of some evidence claims.

This paper examines the role of evidential considerations in relation to pragmatic concerns in statements of group belief, focusing on scientific collaborations that are constituted in part by the aim of evaluating the evidence for scientific claims (evidential collaborations). Drawing upon a case study in high energy particle physics, I seek to show how pragmatic factors that enter into the decision to issue a group statement contribute positively to the epistemic functioning of such groups, contrary to the implications of much of the existing discussion of group belief. I conclude by suggesting that applying social epistemological considerations to scientific collaborations could be practically beneficial, but only if an appropriately broad range of epistemic values is considered.

This book guides readers by gradual steps through the central concepts and debates in the philosophy of science. Using concrete examples from the history of science, Kent W. Staley shows how seemingly abstract philosophical issues are relevant to important aspects of scientific practice. Structured in two parts, the book first tackles the central concepts of the philosophy of science, such as the problem of induction, falsificationism, and underdetermination, and important figures and movements, such as the logical empiricists, Thomas Kuhn, and Paul Feyerabend. The second part turns to contemporary debates in the philosophy of science, such as scientific realism, explanation, the role of values in science, the different views of scientific inference, and probability. This broad yet detailed overview will give readers a strong grounding whilst also providing opportunities for further exploration. It will be of particular interest to students of philosophy, the philosophy of science, and science.

The recent discovery of a Higgs boson prompted increased attention of statisticians and philosophers of science to the statistical methodology of High Energy Physics. Amidst long-standing debates within the field, HEP has adopted a mixed statistical methodology drawing upon both frequentist and Bayesian methods, but with standard frequentist techniques such as significance testing and confidence interval estimation playing a primary role. Physicists within HEP typically deny that their methodological decisions are guided by philosophical convictions, but are instead based on “pragmatic” considerations, thus distancing themselves from what they perhaps perceive as an ongoing pitched ideological battle between frequentists and Bayesians. Here I argue that there is a philosophical orientation to HEP that is neither exclusively frequentist nor Bayesian, but that lies squarely in the tradition of philosophical pragmatism. I further argue that understanding the statistical methodology of HEP through the perspective of pragmatism clarifies the role of and rationale for significance testing in the search for new phenomena such as the Higgs boson.

Many philosophers have claimed that evidence for a theory is better when multiple independent tests yield the same result, i.e., when experimental results are robust. Little has been said about the grounds on which such a claim rests, however. The present essay presents an analysis of the evidential value of robustness that rests on the fallibility of assumptions about the reliability of testing procedures and a distinction between the strength of evidence and the security of an evidence claim. Robustness can enhance the security of an evidence claim either by providing what I call second-order evidence, or by providing back-up evidence for a hypothesis.

Some prominent accounts of scientific evidence treat evidence as an unrelativized concept. But whether belief in a hypothesis is justified seems relative to the epistemic situation of the believer. The issue becomes yet more complicated in the context of group epistemic agents, for then one confronts the problem of relativizing to an epistemic situation that may include conflicting beliefs. As a step toward resolution of these difficulties, an ideal of justification is here proposed that incorporates both an unrelativized evidence requirement and the requirement of the security of the evidence on which a conclusion from data is based. The latter requirement incorporates the consideration of epistemic modal statements.

: Evidence claims depend on fallible assumptions. Three strategies for making true evidence claims in spite of this fallibility are strengthening the support for those assumptions, weakening conclusions, and using multiple independent tests to produce robust evidence. Reliability itself, understood in frequentist terms, does not explain the usefulness of all three strategies; robustness, in particular, sometimes functions in a way that is not well-characterized in terms of reliability. I argue that, in addition to reliability, the security of evidence claims is of epistemic value, where an evidence claim is secure relative to an epistemic situation if it remains true in all scenarios that are epistemically possible relative to that epistemic situation.

I consider the error-statistical account as both a theory of evidence and as a theory of inference. I seek to show how inferences regarding the truth of hypotheses can be upheld by avoiding a certain kind of alternative hypothesis problem. In addition to the testing of assumptions behind the experimental model, I discuss the role of judgments of implausibility. A benefit of my analysis is that it reveals a continuity in the application of error-statistical assessment to low-level empirical hypotheses and highly general theoretical principles. This last point is illustrated with a brief sketch of the issues involved in the parametric framework analysis of tests of physical theories such as General Relativity and of fundamental physical principles such as the Einstein Equivalence Principle.

I contrast two modes of error-elimination relevant to evaluating evidence in accounts that emphasize frequentist reliability. The contrast corresponds to that between the use of of a reliable inference procedure and the critical scrutiny of a procedure with regard to its reliability, in light of what is and is not known about the setting in which the procedure is used. I propose a notion of security as a category of evidential assessment for the latter. In statistical settings, robustness theory and misspecification testing exemplify two distinct strategies for securing statistical inferences.

Evidence claims depend on fallible assumptions. Three strategies for making true evidence claims in spite of this fallibility are strengthening the support for those assumptions, weakening conclusions, and using multiple independent tests to produce robust evidence. Reliability itself, understood in frequentist terms, does not explain the usefulness of all three strategies; robustness, in particular, sometimes functions in a way that is not well-characterized in terms of reliability. I argue that, in addition to reliability, the security of evidence claims is of epistemic value, where an evidence claim is secure relative to an epistemic situation if it remains true in all scenarios that are epistemically possible relative to that epistemic situation.