Matthew Mandelkern

The reasoning abilities of large language models (LLMs) are the topic of a growing body of research in AI and cognitive science. In this paper, we probe the extent to which twenty-nine LLMs are able to distinguish logically correct inferences from logically fallacious ones. We focus on inference patterns involving conditionals (e.g., 'If Ann has a queen, then Bob has a jack') and epistemic modals (e.g., 'Ann might have an ace', 'Bob must have a king'). These inferences have been of special interest to logicians, philosophers, and linguists, since they play a central role in the fundamental human ability to reason about distal possibilities. Assessing LLMs on these inferences is thus highly relevant to the question of how much the reasoning abilities of LLMs match those of humans. All the LLMs we tested make some basic mistakes with conditionals or modals, though zero-shot chain-of-thought prompting helps them make fewer mistakes. Even the best performing LLMs make basic errors in modal reasoning, display logically inconsistent judgments across inference patterns involving epistemic modals and conditionals, and give answers about complex conditional inferences that do not match reported human judgments. These results highlight gaps in basic logical reasoning in today's LLMs.

Recent work has argued that belief is weak: the level of rational credence required for belief is relatively low. That literature has contrasted belief with assertion, arguing that the latter requires an epistemic state much stronger than (weak) belief---perhaps knowledge or even certainty. We argue that this is wrong: assertion is just as weak as belief. We first present a variety of new arguments for this, and then show that the standard arguments for stronger norms are not convincing. Finally, we sketch an alternative picture on which the fundamental norm of assertion is to say what you believe, but both belief and assertion are weak. To help make sense of this, we propose that both belief and assertion involve navigating a tradeoff between accuracy and informativity, and so it can makes sense to believe/say something you only have weak evidence for, if it is informative enough.

In the course of proving a tenability result about the probabilities of conditionals, van Fraassen (1976) introduced a semantics for conditionals based on ω-sequences of worlds, which amounts to a particularly simple special case of ordering semantics for conditionals. On that semantics, ‘If p, then q’ is true at an ω-sequence just in case q is true at the first tail of the sequence where p is true (if such a tail exists). This approach has become increasingly popular in recent years. However, its logic has never been explored. We axiomatize the logic of ω-sequence semantics, showing that it is the result of adding two new axioms to Stalnaker’s logic C2: one, Flattening, which is prima facie attractive, and, and a second, Sequentiality, which is complex and difficult to assess, but, we argue, likely invalid. But we also show that when sequence semantics is generalized from ω-sequences to arbitrary (transfinite) ordinal sequences, the result is a more attractive logic that adds only Flattening to C2. We also explore the logics of a few other interesting restrictions of ordinal sequence semantics. Finally, we address the question of whether sequence semantics is motivated by probabilistic considerations, answering, pace van Fraassen, in the negative.

This paper concerns a controversy between two compelling and popular claims in the theory of ability. One is the claim that ability requires control. The other is the claim that success entails ability, that is, that φ-ing entails that you are able to φ. Since actually φ-ing obviously does not entail that φ is in your control, these two claims cannot both be true. I introduce a new form of evidence to help adjudicate this controversy: judgments about the possibility and probability of ability ascriptions. I argue that these judgments provide evidence in favor of the thesis that success entails ability, and against the thesis that ability requires control. Moreover, I argue that these judgments support an analysis of ability in terms of conditionals.

This essay proposes a new theory of agentive modals: ability modals and their duals, compulsion modals. After criticizing existing approaches—the existential quantificational analysis, the universal quantificational analysis, and the conditional analysis—it presents a new account that builds on both the existential and conditional analyses. On this account, the act conditional analysis, a sentence like ‘John can swim across the river’ says that there is some practically available action that is such that if John tries to do it, he swims across the river. The essay argues that the act conditional analysis avoids the problems faced by existing accounts of agentive modality and shows how the act conditional analysis can be extended to an account of generic agentive modal claims. The upshot is a new vantage point on the role of agentive modal ascriptions in practical discourse: ability ascriptions serve as a kind of hypothetical guarantee, and compulsion ascriptions as a kind of nonhypothetical guarantee.

This paper is about guessing: how people respond to a question when they aren’t certain of the answer. Guesses show surprising and systematic patterns that the most obvious theories don’t explain. We argue that these patterns reveal that people aim to optimize a tradeoff between accuracy and informativity when forming their guess. After spelling out our theory, we use it to argue that guessing plays a central role in our cognitive lives. In particular, our account of guessing yields new theories of belief, assertion, and the conjunction fallacy—the psychological finding that people sometimes rank a conjunction as more probable than one of its conjuncts. More generally, we suggest that guessing helps explain how boundedly rational agents like us navigate a complex, uncertain world.

The Identity principle says that conditionals with the form 'If p, then p' are logical truths. Identity is overwhelmingly plausible, and has rarely been explicitly challenged. But a wide range of conditionals nonetheless invalidate it. I explain the problem, and argue that the culprit is the principle known as Import-Export, which we must thus reject. I then explore how we can reject Import-Export in a way that still makes sense of the intuitions that support it, arguing that the differences between indicative and subjunctive conditionals play a key role in solving this puzzle.