Teddy Seidenfeld

Can there be good reasons for judging one set of probabilistic assertions more reliable than a second? There are many candidates for measuring "goodness" of probabilistic forecasts. Here, I focus on one such aspirant: calibration. Calibration requires an alignment of announced probabilities and observed relative frequency, e.g., 50 percent of forecasts made with the announced probability of.5 occur, 70 percent of forecasts made with probability.7 occur, etc. To summarize the conclusions: (i) Surveys designed to display calibration curves, from which a recalibration is to be calculated, are useless without due consideration for the interconnections between questions (forecasts) in the survey. (ii) Subject to feedback, calibration in the long run is otiose. It gives no ground for validating one coherent opinion over another as each coherent forecaster is (almost) sure of his own long-run calibration. (iii) Calibration in the short run is an inducement to hedge forecasts. A calibration score, in the short run, is improper. It gives the forecaster reason to feign violation of total evidence by enticing him to use the more predictable frequencies in a larger finite reference class than that directly relevant

It has long been known that the practice of testing all hypotheses at the same level , regardless of the distribution of the data, is not consistent with Bayesian expected utility maximization. According to de Finetti’s “Dutch Book” argument, procedures that are not consistent with expected utility maximization are incoherent and they lead to gambles that are sure to lose no matter what happens. In this paper, we use a method to measure the rate at which incoherent procedures are sure to lose, so that we can distinguish slightly incoherent procedures from grossly incoherent ones. We present an analysis of testing a simple hypothesis against a simple alternative as a case‐study of how the method can work

For Savage (1954) as for de Finetti (1974), the existence of subjective (personal) probability is a consequence of the normative theory of preference. (De Finetti achieves the reduction of belief to desire with his generalized Dutch-Book argument for Previsions.) Both Savage and de Finetti rebel against legislating countable additivity for subjective probability. They require merely that probability be finitely additive. Simultaneously, they insist that their theories of preference are weak, accommodating all but self-defeating desires. In this paper we dispute these claims by showing that the following three cannot simultaneously hold: (i) Coherent belief is reducible to rational preference, i.e. the generalized Dutch-Book argument fixes standards of coherence. (ii) Finitely additive probability is coherent. (iii) Admissible preference structures may be free of consequences, i.e. they may lack prizes whose values are robust against all contingencies

Several axiom systems for preference among acts lead to a unique probability and a state-independent utility such that acts are ranked according to their expected utilities. These axioms have been used as a foundation for Bayesian decision theory and subjective probability calculus. In this article we note that the uniqueness of the probability is relative to the choice of whatcounts as a constant outcome. Although it is sometimes clear what should be considered constant, in many cases there are several possible choices. Each choice can lead to a different "unique" probability and utility. By focusing attention on statedependent utilities, we determine conditions under which a truly unique probability and utility can be determined from an agent's expressed preferences among acts. Suppose that an agent's preference can be represented in terms of a probability P and a utility U.That is, the agent prefers one act to another iff the expected utility of that act is higher than that of the other. There are many other equivalent representations in terms of probabilities Q, which are mutually absolutely continuous with P, and state-dependent utilities V, which differ from U by possibly different positive affine transformations in each state of nature. We describe an example in which there are two different but equivalent state-independent utility representations for the same preference structure. They differ in which acts count as constants. The acts involve receiving different amounts of one or the other of two currencies, and the states are different exchange rates between the currencies. It is easy to see how it would not be possible for constant amounts of both currencies to have simultaneously constant values across the differentstates. Savage (1954, sec. 5.5) discovered a situation in which two seemingly equivalent preference structures are represented by different pairs of probability and utility. He attributed the phenomenon to the construction of a "small world." We show that the small world problem is just another example of two different, but equivalent, representations treating different actsas constants. Finally, we prove a theorem (similar to one of Karni 1985) that shows how to elicit a unique state-dependent utility and does not assume that there are prizes with constant value. To do this, we define a new hypothetical kind of act in which both the prize to be awarded and the state of nature are determined by an auxiliary experiment

Statistical decision theory, whether based on Bayesian principles or other concepts such as minimax or admissibility, relies on minimizing expected loss or maximizing expected utility. Loss and utility functions are generally treated as unit-less numerical measures of value for consequences. Here, we address the issue of the units in which loss and utility are settled and the implications that those units have on the rankings of potential decisions. When multiple currencies are available for paying the loss, one must take explicit account of which currency is used as well as the exchange rates between the various available currencies

The degree of incoherence, when previsions are not made in accordance with a probability measure, is measured by either of two rates at which an incoherent bookie can be made a sure loser. Each bet is considered as an investment from the points of view of both the bookie and a gambler who takes the bet. From each viewpoint, we define an amount invested (or escrowed) for each bet, and the sure loss of incoherent previsions is divided by the escrow to determine the rate of incoherence. Potential applications include the treatment of arbitrage opportunities in financial markets and the degree of incoherence of classical statistical procedures. We illustrate the latter with the example of hypothesis testing at a fixed size.

It has long been known that the practice of testing all hypotheses at the same level , regardless of the distribution of the data, is not consistent with Bayesian expected utility maximization. According to de Finetti’s “Dutch Book” argument, procedures that are not consistent with expected utility maximization are incoherent and they lead to gambles that are sure to lose no matter what happens. In this paper, we use a method to measure the rate at which incoherent procedures are sure to lose, so that we can distinguish slightly incoherent procedures from grossly incoherent ones. We present an analysis of testing a simple hypothesis against a simple alternative as a case‐study of how the method can work

Experimenters sometimes insist that it is unwise to examine data before determining how to analyze them, as it creates the potential for biased results. I explore the rationale behind this methodological guideline from the standpoint of an error statistical theory of evidence, and I discuss a method of evaluating evidence in some contexts when this predesignation rule has been violated. I illustrate the problem of potential bias, and the method by which it may be addressed, with an example from the search for the top quark. A point in favor of the error statistical theory is its ability, demonstrated here, to explicate such methodological problems and suggest solutions, within the framework of an objective theory of evidence.

When can a Bayesian investigator select an hypothesis H and design an experiment (or a sequence of experiments) to make certain that, given the experimental outcome(s), the posterior probability of H will be lower than its prior probability? We report an elementary result which establishes sufficient conditions under which this reasoning to a foregone conclusion cannot occur. Through an example, we discuss how this result extends to the perspective of an onlooker who agrees with the investigator about the statistical model for the data but who holds a different prior probability for the statistical parameters of that model. We consider, specifically, one-sided and two-sided statistical hypotheses involving i.i.d. Normal data with conjugate priors. In a concluding section, using an "improper" prior, we illustrate how the preceding results depend upon the assumption that probability is countably additive

This special issue of the International Journal of Approximate Reasoning grew out of the 8th International Symposium on Imprecise Probability: Theories and Applications. The symposium was organized by the Society for Imprecise Probability: Theories and Applications at the Université de Technologie de Compiègne in July 2013. The biennial ISIPTA meetings are well established among international conferences on generalized methods for uncertainty quantification. The first ISIPTA took place in Gent in 1999, followed by meetings in Cornell, Lugano, Carnegie Mellon, Prague, Durham and Innsbruck. Compiègne proved to be a very nice location for ISIPTA 2013, offering wonderful opportunities for collaborations and discussions, as well as sightseeing places such as its imperial palace.

Conditioning can make imprecise probabilities uniformly more imprecise. We call this effect "dilation". In a previous paper (1993), Seidenfeld and Wasserman established some basic results about dilation. In this paper we further investigate dilation on several models. In particular, we consider conditions under which dilation persists under marginalization and we quantify the degree of dilation. We also show that dilation manifests itself asymptotically in certain robust Bayesian models and we characterize the rate at which dilation occurs

We report two issues concerning diverging sets of Bayesian (conditional) probabilities-divergence of "posteriors"-that can result with increasing evidence. Consider a set P of probabilities typically, but not always, based on a set of Bayesian "priors." Fix E, an event of interest, and X, a random variable to be observed. With respect to P, when the set of conditional probabilities for E, given X, strictly contains the set of unconditional probabilities for E, for each possible outcome X = x, call this phenomenon dilation of the set of probabilities (Seidenfeld and Wasserman 1993). Thus, dilation contrasts with the asymptotic merging of posterior probabilities reported by Savage (1954) and by Blackwell and Dubins (1962). (1) In a wide variety of models for Robust Bayesian inference the extent to which X dilates E is related to a model specific index of how far key elements of P are from a distribution that makes X and E independent. (2) At a fixed confidence level, (1-α), Classical interval estimates A n for, e.g., a Normal mean θ have length O(n -1/2 ) (for sample size n). Of course, the confidence level correctly reports the (prior) probability that θ ∈ A n ,P(A n )=1-α , independent of the prior for θ . However, as shown by Pericchi and Walley (1991), if an ε -contamination class is used for the prior on the parameter θ , there is asymptotic (posterior) dilation for the A n , given the data. If, however, the intervals A ′ n are chosen with length $O(\sqrt{\log (\text{n})/\text{n})}$ , then there is no asymptotic dilation. We discuss the asymptotic rates of dilation for ClassClassical and Bayesian interval estimates and relate these to Bayesian hypothesis testing.

The Sleeping Beauty problem has spawned a debate between “Thirders” and “Halfers” who draw conflicting conclusions about Sleeping Beauty’s credence that a coin lands Heads. Our analysis is based on a probability model for what Sleeping Beauty knows at each time during the Experiment. We show that conflicting conclusions result from different modeling assumptions that each group makes. Our analysis uses a standard “Bayesian” account of rational belief with conditioning. No special handling is used for self-locating beliefs or centered propositions. We also explore what fair prices Sleeping Beauty computes for gambles that she might be offered during the Experiment.