Gregor Betz

Argument analysis is a powerful tool for structuring policy deliberation and decision-making, especially when complexity and uncertainty loom large. Argument analysis seeks to determine which claims are justified or criticized by a given argumentation, how strong an argument is, on which implicit assumptions it rests, how it relates to other arguments in a controversy, and which standpoints one can reasonably adopt in view of a given state of debate. This chapter first gives an overview of the activities involved in argument analysis and discusses the various aims that guide argument analysis. It then introduces methods for reconstructing and evaluating individual arguments as well as complex argumentation and debates. In their application to decisions under great uncertainty, these methods help to identify coherent positions, to discern important points of (dis)agreement, as well as to avoid spurious consensus and oversimplification.

Reflective equilibrium ("RE", for short) is a method of justification which works roughly as follows: We start with our pre-theoretical judgements (about, e.g. moral issues) and try to explain them by a systematic theory. This leads to a process in which judgements and principles are mutually adjusted to each other until a state of equilibrium is reached. For more than half a century, RE has been very popular, as well as controversial, among philosophers of many persuasions. Given how frequently the method is mentioned, embraced and attacked, it is surprising that only very few attempts have been made at analysing and developing RE in detail. This topical collection brings together papers that develop RE on a conceptual level, analyse it using formal methods or investigate applications of RE.

This paper develops a natural-language agent-based model of argumentation (ABMA). Its artificial deliberative agents (ADAs) are constructed with the help of so-called neural language models recently developed in AI and computational linguistics. ADAs are equipped with a minimalist belief system and may generate and submit novel contributions to a conversation. The natural-language ABMA allows us to simulate collective deliberation in English, i.e. with arguments, reasons, and claims themselves — rather than with their mathematical representations (as in symbolic models). This paper uses the natural-language ABMA to test the robustness of symbolic reason-balancing models of argumentation (Mäs & Flache 2013; Singer et al. 2019): First of all, as long as ADAs remain passive, confirmation bias and homophily updating trigger polarization, which is consistent with results from symbolic models. However, once ADAs start to actively generate new contributions, the evolution of a conversation is dominated by properties of the agents as authors. This suggests that the creation of new arguments, reasons, and claims critically affects a conversation and is of pivotal importance for understanding the dynamics of collective deliberation. The paper closes by pointing out further fruitful applications of the model and challenges for future research.

This paper takes a first step towards a critical thinking curriculum for neural auto-regressive language models. We introduce a synthetic corpus of deductively valid arguments, and generate artificial argumentative texts to train and evaluate GPT-2. Significant transfer learning effects can be observed: Training a model on three simple core schemes allows it to accurately complete conclusions of different, and more complex types of arguments, too. The language models generalize the core argument schemes in a correct way. Moreover, we obtain consistent and promising results for NLU benchmarks. In particular, pre-training on the argument schemes raises zero-shot accuracy on the GLUE diagnostics by up to 15 percentage points. The findings suggest that intermediary pre-training on texts that exemplify basic reasoning abilities (such as typically covered in critical thinking textbooks) might help language models to acquire a broad range of reasoning skills. The synthetic argumentative texts presented in this paper are a promising starting point for building such a "critical thinking curriculum for language models."

It is argued that suitably trained neural language models exhibit key properties of epistemic agency: they hold probabilistically coherent and logically consistent degrees of belief, which they can rationally revise in the face of novel evidence. To this purpose, we conduct computational experiments with rankers: T5 models [Raffel et al. 2020] that are pretrained on carefully designed synthetic corpora. Moreover, we introduce a procedure for eliciting a model’s degrees of belief, and define numerical metrics that measure the extent to which given degrees of belief violate (probabilistic, logical, and Bayesian) rationality constraints. While pretrained rankers are found to suffer from global inconsistency (in agreement with, e.g., [Jang et al. 2021]), we observe that subsequent self-training on auto-generated texts allows rankers to gradually obtain a probabilistically coherent belief system that is aligned with logical constraints. In addition, such self-training is found to have a pivotal role in rational evidential learning, too, for it seems to enable rankers to propagate a novel evidence item through their belief systems, successively re-adjusting individual degrees of belief. All this, we conclude, confirms the Rationality Hypothesis, i.e., the claim that suitable trained NLMs may exhibit advanced rational skills. We suggest that this hypothesis has empirical, yet also normative and conceptual ramifications far beyond the practical linguistic problems NLMs have originally been designed to solve.

Neural language models (NLMs) are susceptible to producing inconsistent output. This paper proposes a new diagnosis as well as a novel remedy for NLMs' incoherence. We train NLMs on synthetic text corpora that are created by simulating text production in a society. For diagnostic purposes, we explicitly model the individual belief systems of artificial agents (authors) who produce corpus texts. NLMs, trained on those texts, can be shown to aggregate the judgments of individual authors during pre-training according to sentence-wise vote ratios (roughly, reporting frequencies), which inevitably leads to so-called discursive dilemmas: aggregate judgments are inconsistent even though all individual belief states are consistent. As a remedy for such inconsistencies, we develop a self-training procedure—inspired by the concept of reflective equilibrium—that effectively reduces the extent of logical incoherence in a model's belief system, corrects global mis-confidence, and eventually allows the model to settle on a new, epistemically superior belief state. Thus, social choice theory helps to understand why NLMs are prone to produce inconsistencies; epistemology suggests how to get rid of them.

How do opinion diversity and belief polarisation affect epistemic group decision-making, particularly if decisions must be made without delay and on the basis of permissive evidence? In an agent-based model, we track the consistency of group opinions aggregated through sentence-wise majority voting. Simulations on the model reveal that high opinion diversity, but not polarisation, incurs a significant inconsistency risk. These results indicate that epistemic group decisions based on permissive evidence can be particularly difficult for diverse groups. The results also improve our understanding of what can reasonably be expected of expert groups, and where expert advice might have limits.

Reflective equilibrium (RE) is often regarded as a powerful method in ethics, logic, and even philosophy in general. Despite this popularity, characterizations of the method have been fairly vague and unspecific so far. It thus may be doubted whether RE is more than a jumble of appealing but ultimately sketchy ideas that cannot be spelled out consistently. In this paper, we dispel such doubts by devising a formal model of RE. The model contains as components the agent’s commitments and a theory that tries to systematize the commitments. It yields a precise picture of how the commitments and the theory are adjusted to each other. The model differentiates between equilibrium as a target state and the dynamic equilibration process. First solutions to the model, obtained by computer simulation, show that the method allows for consistent specification and that the model’s implications are plausible in view of expectations on RE. In particular, the mutual adjustment of commitments and theory can improve one’s commitments, as proponents of RE have suggested. We argue that our model is fruitful not only because it points to issues that need to be dealt with for a better understanding of RE, but also because it provides the means to address these issues.

There are different varieties of conservatism concerning belief formation and revision. We assesses the veritistic effects of a particular kind of conservatism commonly attributed to Quine: the so-called maxim of minimum mutiliation, which states that agents should give up as few beliefs as possible when facing recalcitrant evidence. Based on a formal bounded rationality model of belief revision, which parametrizes degree of conservatism, and corresponding multi-agent simulations, we eventually argue against doxastic conservatism from the vantage point of veritistic social epistemology.

If we want to understand how extremist group ideologies are established, we have to comprehend the social processes which form the basis of the emergence and distribution of such beliefs. In our chapter, we present an innovative approach to examining these processes and explaining how they function: with the method of computer-based simulation of opinion formation, we develop heuristic explanatory models which help to generate new and interesting hypotheses. The focus is thereby not on individuals and their idiosyncrasies but on the dynamic mutual adaptation of beliefs in a group. These dynamics can produce an incremental establishment of “charismatic” opinion leaders and an increasing radicalization and alienation. A prototype of such a simulation model has produced promising first results which are presented and discussed.

Philosophy of science attempts to reconstruct science as a rational cognitive enterprise. In doing so, it depicts a normative ideal of knowledge acquisition and does not primarily seek to describe actual scientific practice in an empirically adequate way. A comprehensive picture of what good science consists in may serve as a standard against which we evaluate and criticize actual scientific practices. Such a normative picture may also explain why it is reasonable for us to trust scientists – to the extent that they live up to the ideal – and to rely on their findings in decision making. Likewise, a sound normative understanding of science exposes the limits of scientific understanding and prevents us from placing blind faith in scientists and experts. For these reasons, philosophy of science represents a useful resource and background theory for the practice and study of science communication. In this handbook article, we provide an opinionated introduction to philosophy of science by flashing a light on 22 central issues which (we think) are of special interest to scholars and practitioners of science communication – and, in particular, to scholars and practitioners of external science communication.

Intended as a practical guide for decision analysts, this chapter provides an introduction to reasoning under great uncertainty. It seeks to incorporate standard methods of risk analysis in a broader argumentative framework by re-interpreting them as specific (consequentialist) arguments that may inform a policy debate—side by side along further (possibly non-consequentialist) arguments which standard economic analysis does not account for. The first part of the chapter reviews arguments that can be advanced in a policy debate despite deep uncertainty about policy outcomes, i.e. arguments which assume that uncertainties surrounding policy outcomes cannot be (probabilistically) quantified. The second part of the chapter discusses the epistemic challenge of reasoning under great uncertainty, which consists in identifying all possible outcomes of the alternative policy options. It is argued that our possibilistic foreknowledge should be cast in nuanced terms and that future surprises—triggered by major flaws in one’s possibilistic outlook—should be anticipated in policy deliberation.

This paper presents OpMAP: a tool for visualizing large scale, multi-dimensional opinion spaces as geographic maps. OpMAP represents opinions as labelings on a structured deductive argumentation framework. It uses probabilistic degrees of justification and Bayesian coherence measures to calculate how strongly any two opinions cohere with each other. The opinion sample is, accordingly, represented as a weighted graph, a so-called opinion graph, with opinion vectors serving as nodes and coherence values as edge weights. OpMAP partitions the nodes of the opinion graph by using clustering methods. Finally, the graph is visualized as a geographic map using a method based on a particular (e.g., force-directed) layout.

Climate policy decisions are decisions under uncertainty and are, therefore, based on a range of future climate scenarios, describing possible consequences of alternative policies. Accordingly, the methodology for setting up such a scenario range becomes pivotal in climate policy advice. The preferred methodology of the Intergovernmental Panel on Climate Change will be characterised as,,modal verificationism"; it suffers from severe shortcomings which disqualify it for scientific policy advice. Modal falsificationism, as a more sound alternative, would radically alter the way the climate scenario range is set up. Climate science's inability to find robust upper bounds for future temperature rise in line with modal falsificationism does not disprove that methodology, rather, this very fact prescribes even more drastic efforts to curb CO2 emissions than currently proposed.

This paper gives an explication of our intuitive notion of strength of justification in a controversial debate. It defines a thesis' degree of justification within the bipolar argumentation framework of the theory of dialectical structures as the ratio of coherently adoptable positions according to which that thesis is true over all coherently adoptable positions. Broadening this definition, the notion of conditional degree of justification, i.e.\ degree of partial entailment, is introduced. Thus defined degrees of justification correspond to our pre-theoretic intuitions in the sense that supporting and defending a thesis t increases, whereas attacking it decreases, t's degree of justification. Moreover, it is shown that (conditional) degrees of justification are (conditional) probabilities. Eventually, the paper explains that it is rational to believe theses with a high degree of justification insofar as this strengthens the robustness of one's position.

This paper develops concepts and procedures for the evaluation of complex debates. They provide means for answering such questions as whether a thesis has to be considered as proven or disproven in a debate or who carries a burden of proof. While being based on classical logic, this framework represents an (argument-based) approach to non-monotonic, or defeasible reasoning. Debates are analysed as dialectical structures, i.e. argumentation systems with an attack- as well as a support-relationship. The recursive status assignment over the arguments is conditionalised on proponents in a debate. The problem of multiple status assignments arising on circular structures is solved by showing that uniqueness can be guaranteed qua reconstruction of a debate. The notion of burden of proof as well as other discursive aims rational proponents pursue in a debate is defined within the framework.