Katie Steele

We argue that concerns about double-counting—using the same evidence both to calibrate or tune climate models and also to confirm or verify that the models are adequate—deserve more careful scrutiny in climate modelling circles. It is widely held that double-counting is bad and that separate data must be used for calibration and confirmation. We show that this is far from obviously true, and that climate scientists may be confusing their targets. Our analysis turns on a Bayesian/relative-likelihood approach to incremental confirmation. According to this approach, double-counting is entirely proper. We go on to discuss plausible difficulties with calibrating climate models, and we distinguish more and less ambitious notions of confirmation. Strong claims of confirmation may not, in many cases, be warranted, but it would be a mistake to regard double-counting as the culprit. 1 Introduction2 Remarks about Models and Adequacy-for-Purpose3 Evidence for Calibration Can Also Yield Comparative Confirmation3.1 Double-counting I3.2 Double-counting II4 Climate Science Examples: Comparative Confirmation in Practice4.1 Confirmation due to better and worse best fits4.2 Confirmation due to more and less plausible forcings values5 Old Evidence6 Doubts about the Relevance of Past Data7 Non-comparative Confirmation and Catch-Alls8 Climate Science Example: Non-comparative Confirmation and Catch-Alls in Practice9 Concluding Remarks

Richard Rudner famously argues that the communication of scientific advice to policy makers involves ethical value judgments. His argument has, however, been rightly criticized. This article revives Rudner’s conclusion, by strengthening both his lines of argument: we generalize his initial assumption regarding the form in which scientists must communicate their results and complete his ‘backup’ argument by appealing to the difference between private and public decisions. Our conclusion that science advisors must, for deep-seated pragmatic reasons, make value judgments is further bolstered by reflections on how the scientific contribution to policy is far less straightforward than the Rudner-style model suggests

I focus my discussion on the well-known Ellsberg paradox. I find good normative reasons for incorporating non-precise belief, as represented by sets of probabilities, in an Ellsberg decision model. This amounts to forgoing the completeness axiom of expected utility theory. Provided that probability sets are interpreted as genuinely indeterminate belief, such a model can moreover make the “Ellsberg choices” rationally permissible. Without some further element to the story, however, the model does not explain how an agent may come to have unique preferences for each of the Ellsberg options. Levi holds that the extra element amounts to innocuous secondary “risk” or security considerations that are used to break ties when more than one option is rationally permissible. While I think a lexical choice rule of this kind is very plausible, I argue that it involves a greater break with xpected utility theory than mere violation of the ordering axiom

In engineering, as in other scientific fields, researchers seek to confirm their models with real-world data. It is common practice to assess models in terms of the distance between the model outputs and the corresponding experimental observations. An important question that arises is whether the model should then be ‘tuned’, in the sense of estimating the values of free parameters to get a better fit with the data, and furthermore whether the tuned model can be confirmed with the same data used to tune it. This dual use of data is often disparagingly referred to as ‘double-counting’. Here, we analyse these issues, with reference to selected research articles in engineering. Our example studies illustrate more and less controversial practices of model tuning and double-counting, both of which, we argue, can be shown to be legitimate within a Bayesian framework. The question nonetheless remains as to whether the implied scientific assumptions in each case are apt from the engineering point of view

This paper considers a puzzling conflict between two positions that are each compelling: it is irrational for an agent to pay to avoid `free' evidence before making a decision, and rational agents may have imprecise beliefs and/or desires. Indeed, we show that Good's theorem concerning the invariable choice-worthiness of free evidence does not generalise to the imprecise realm, given the plausible existing decision theories for handling imprecision. A key ingredient in the analysis, and a potential source of controversy, is the general approach taken for resolving sequential decision problems { we make explicit what the key alternatives are and defend our own approach. Furthermore, we endorse a resolution of the aforementioned puzzle { we privilege decision theories that merely permit avoiding free evidence over decision theories for which avoiding free evidence is uniquely admissible. Finally, we situate this particular result about free evidence within the broader `dynamic-coherence' literature.