Gregory Wheeler

Analyzing geometric properties of high-dimensional loss functions, such as local curvature and the existence of other optima around a certain point in loss space, can help provide a better understanding of the interplay between neural network structure, implementation attributes, and learning performance. In this work, we combine concepts from high-dimensional probability and differential geometry to study how curvature properties in lower-dimensional loss representations depend on those in the original loss space. We show that saddle points in the original space are rarely correctly identified as such in expected lower-dimensional representations if random projections are used. The principal curvature in the expected lower-dimensional representation is proportional to the mean curvature in the original loss space. Hence, the mean curvature in the original loss space determines if saddle points appear, on average, as either minima, maxima, or almost flat regions. We use the connection between expected curvature in random projections and mean curvature in the original space (i.e., the normalized Hessian trace) to compute Hutchinson-type trace estimates without calculating Hessian-vector products as in the original Hutchinson method. Because random projections are not suitable to correctly identify saddle information, we propose to study projections along dominant Hessian directions that are associated with the largest and smallest principal curvatures. We connect our findings to the ongoing debate on loss landscape flatness and generalizability. Finally, for different common image classifiers and a function approximator, we show and compare random and Hessian projections of loss landscapes with up to about 7×10^6 parameters.

Analyzing geometric properties of high-dimensional loss functions, such as local curvature and the existence of other optima around a certain point in loss space, can help provide a better understanding of the interplay between neural network structure, implementation attributes, and learning performance. In this work, we combine concepts from high-dimensional probability and differential geometry to study how curvature properties in lower-dimensional loss representations depend on those in the original loss space. We show that saddle points in the original space are rarely correctly identified as such in lower-dimensional representations if random projections are used. In such projections, the expected curvature in a lower-dimensional representation is proportional to the mean curvature in the original loss space. Hence, the mean curvature in the original loss space determines if saddle points appear, on average, as either minima, maxima, or almost flat regions. We use the connection between expected curvature and mean curvature (i.e., the normalized Hessian trace) to estimate the trace of Hessians without calculating the Hessian or Hessian-vector products as in Hutchinson’s method. Because random projections are not able to correctly identify saddle information, we propose to study projections along Hessian directions that are associated with the largest and smallest principal curvatures. We connect our findings to the ongoing debate on loss landscape flatness and generalizability. Finally, we illustrate our method in numerical experiments on different image classifiers with up to about 7 × 10^6 parameters.

This festschrift for Teddy Seidenfeld is a collection of newly commissioned essays on imprecise probability by leading experts in the field. Each contribution touches on Teddy’s seminal contributions to the field of imprecise probability, and gives an up-to-date state of the field that cannot be found elsewhere. The title, “a reflection on the foundations of probability and statistics”, calls back to Teddy’s seminal book, “Rethinking the Foundations of Probability and Statistics”, bookends to a career that has made fundamental contributions to the foundations of probability and statistics, and shaped the field of imprecise probability. It is aimed at scholars engaged in probability and statistics.

The field of of imprecise probability has matured, in no small part because of Teddy Seidenfeld’s decades of original scholarship and essential contributions to building and sustaining the ISIPTA community. Although the basic idea behind imprecise probability is (at least) 150 years old, a mature mathematical theory has only taken full form in the last 30 years. Interest in imprecise probability during this period has also grown, but many of the ideas that the mature theory serves can be difficult to apprehend to those new to the subject. Although these fundamental ideas are common knowledge in the ISIPTA community, they are expressed, when they are expressed at all, obliquely, over the course of years with students and colleagues. A single essay cannot convey the store of common knowledge from any research community, let alone the ISIPTA community. But, this essay nevertheless is an attempt to guide those familiar with the basic Bayesian framework to appreciate some of the elegant and powerful ideas that underpin the contemporary theory of lower previsions, which is the theory that most people associate with the term ‘imprecise probabilities’.

The desirable gambles framework offers the most comprehensive foundations for the theory of lower pre- visions, which in turn affords the most general ac- count of imprecise probabilities. Nevertheless, for all its generality, the theory of lower previsions rests on the notion of linear utility. This commitment to linearity is clearest in the coherence axioms for sets of desirable gambles. This paper considers two routes to relaxing this commitment. The first preserves the additive structure of the desirable gambles framework and the machinery for coherent inference but detaches the interpretation of desirability from the multiplicative scale invariance axiom. The second strays from the additive combination axiom to accommodate repeated gambles that return rewards by a non-stationary processes that is not necessarily additive. Unlike the first approach, which is a conservative amendment to the desirable gambles framework, the second is a rad- ical departure. Yet, common to both is a method for describing rewards called discounted utility.

Recent debate about the error theory has taken a ‘formal turn’. On the one hand, there are those who argue that the error theory should be rejected because of its difficulties in providing a convincing formal account of the logic and semantics of moral claims. On the other hand, there are those who claim that such formal objections fail, maintaining that arguments against the error theory must be of a substantive rather than a formal kind. In this paper, we argue that formal objections to the error theory cannot be eschewed but must be met head-on.

Traditionally, logic has been the dominant formal method within philosophy. Are logical methods still dominant today, or have the types of formal methods used in philosophy changed in recent times? To address this question, we coded a sample of philosophy papers from the late 2000s and from the late 2010s for the formal methods they used. The results indicate that the proportion of papers using logical methods remained more or less constant over that time period but the proportion of papers using probabilistic methods was approximately three times higher in the late 2010s than it was in the late 2000s. Further analyses explored this change by looking more closely at specific methods, specific levels of technical engagement, and specific subdisciplines within philosophy. These analyses indicate that the increasing proportion of papers using probabilistic methods was pervasive, not confined to particular probabilistic methods, levels of sophistication, or subdisciplines.

The theory of lower previsions is designed around the principles of coherence and sure-loss avoidance, thus steers clear of all the updating anomalies highlighted in Gong and Meng's "Judicious Judgment Meets Unsettling Updating: Dilation, Sure Loss, and Simpson's Paradox" except dilation. In fact, the traditional problem with the theory of imprecise probability is that coherent inference is too complicated rather than unsettling. Progress has been made simplifying coherent inference by demoting sets of probabilities from fundamental building blocks to secondary representations that are derived or discarded as needed.

Conditionals and conditional reasoning have been a long-standing focus of research across a number of disciplines, ranging from psychology through linguistics to philosophy. But almost no work has concerned itself with the question of how hearing or reading a conditional changes our beliefs. Given that we acquire much—perhaps most—of what we believe through the testimony of others, the simple matter of acquiring conditionals via others’ assertion of a conditional seems integral to any full understanding of the conditional and conditional reasoning. In this paper we detail a number of basic intuitions about how beliefs might change in response to a conditional being uttered, and show how these are backed by behavioral data. In the remainder of the paper, we then show how these deceptively simple phenomena pose a fundamental challenge to present theoretical accounts of the conditional and conditional reasoning – a challenge which no account presently fully meets.

In this paper, we present a new semantic challenge to the moral error theory. Its first component calls upon moral error theorists to deliver a deontic semantics that is consistent with the error-theoretic denial of moral truths by returning the truth-value false to all moral deontic sentences. We call this the ‘consistency challenge’ to the moral error theory. Its second component demands that error theorists explain in which way moral deontic assertions can be seen to differ in meaning despite necessarily sharing the same intension. We call this the ‘triviality challenge’ to the moral error theory. Error theorists can either meet the consistency challenge or the triviality challenge, we argue, but are hard pressed to meet both.

This paper presents the progicnet programme. It proposes a general framework for probabilistic logic that can guide inference based on both logical and probabilistic input. After an introduction to the framework as such, it is illustrated by means of a toy example from psychometrics. It is shown that the framework can accommodate a number of approaches to probabilistic reasoning: Bayesian statistical inference, evidential probability, probabilistic argumentation, and objective Bayesianism. The framework thus provides insight into the relations between these approaches, it illustrates how the results of different approaches can be combined, and it provides a basis for doing efficient inference in each of the approaches.

Coherentism maintains that coherent beliefs are more likely to be true than incoherent beliefs, and that coherent evidence provides more confirmation of a hypothesis when the evidence is made coherent by the explanation provided by that hypothesis. Although probabilistic models of credence ought to be well-suited to justifying such claims, negative results from Bayesian epistemology have suggested otherwise. In this essay we argue that the connection between coherence and confirmation should be understood as a relation mediated by the causal relationships among the evidence and a hypothesis, and we offer a framework for doing so by fitting together probabilistic models of coherence, confirmation, and causation. We show that the causal structure among the evidence and hypothesis is sometimes enough to determine whether the coherence of the evidence boosts confirmation of the hypothesis, makes no difference to it, or even reduces it. We also show that, ceteris paribus, it is not the coherence of the evidence that boosts confirmation, but rather the ratio of the coherence of the evidence to the coherence of the evidence conditional on a hypothesis

No one has a well developed solution to Duhem's problem, the problem of how experimental evidence warrants revision of our theories. Deborah Mayo proposes a solution to Duhem's problem in route to her more ambitious program of providing a philosophical account of inductive inference and experimental knowledge. This paper is a response to Mayo's Error Statistics (ES) program, paying particular attention to her response to Duhem's problem. It turns out that Mayo's purported solution to Duhem's problem is very significant to her project, for the epistemic license claimed by ES and the philosophical underpinnings to her account of experimental knowledge depend on this solution. By introducing the partition problem, I argue that ES fails to solve Duhem's problem and therefore fails to provide an adequate account of experimental knowledge

A sound and complete axiomatization of two philosophy tabloids is given, Leiter Logic (KB) and Deontic Leiter Logic (KDB), in single agent format, the latter of which can be extended to Shame Game Logic for multiple players. The (B) schema captures the mechanism of tabloid inference, illustrating the perils of interpreting a provability operator as an epistemic modal. To mark this hazard, and to preserve Brouwer's good name, the (B) schema interpreted to govern epistemic modals should be called The Blog Schema.

Classical modal logics, based on the neighborhood semantics of Scott and Montague, provide a generalization of the familiar normal systems based on Kripke semantics. This paper defines AGM revision operators on several first-order monotonic modal correspondents, where each first-order correspondence language is defined by Marc Pauly’s version of the van Benthem characterization theorem for monotone modal logic. A revision problem expressed in a monotone modal system is translated into first-order logic, the revision is performed, and the new belief set is translated back to the original modal system. An example is provided for the logic of Risky Knowledge that uses modal AGM contraction to construct counter-factual evidence sets in order to investigate robustness of a probability assignment given some evidence set. A proof of correctness is given

Two compelling principles, the Reasonable Range Principle and the Preservation of Irrelevant Evidence Principle, are necessary conditions that any response to peer disagreements ought to abide by. The Reasonable Range Principle maintains that a resolution to a peer disagreement should not fall outside the range of views expressed by the peers in their dispute, whereas the Preservation of Irrelevant Evidence Principle maintains that a resolution strategy should be able to preserve unanimous judgments of evidential irrelevance among the peers. No standard Bayesian resolution strategy satisfies the PIE Principle, however, and we give a loss aversion argument in support of PIE and against Bayes. The theory of imprecise probability allows one to satisfy both principles, and we introduce the notion of a set-based credal judgment to frame and address a range of subtle issues that arise in peer disagreements.