Rosa W. Runhardt

In this thesis I investigate causal inquiry in the social sciences, drawing on examples from various disciplines and in particular from conflict studies. In a backlash against the pervasiveness of statistical methods, in the last decade certain social scientists have focused on finding the causal mechanisms behind observed correlations. To provide evidence for such mechanisms, researchers increasingly rely on ‘process tracing’, a method which attempts to give evidence for causal relations by specifying the chain of events connecting a putative cause and effect of interest. I will ask whether the causal claims process tracers make are defensible, and where they are not defensible I will ask how we can improve the method. Throughout these investigations, I show that the conclusions of process tracing (and indeed ofthe social sciences more generally) are constrained both by the causal structure ofthe social world and by social scientists’ aims and values. My central argument is this: all instances of social phenomena have causally relevant differences, which implies that any research design that requires some comparison between cases (like process tracing) is limited by how we systematize these phenomena. Moreover, such research cannot rely on stable regularities. Nevertheless, to forego causal conclusions altogether is not the right response to these limitations; by carefully outlining our epistemic assumptions we can make progress in causal inquiry. While I use philosophical theories of causation to comment on the feasibility of a social scientific method, I also do the reverse: by investigating a popular contemporary method in the social sciences, I show to what extent our philosophical theories of causation are workable in practice. Thus, this thesis is both a methodological and a philosophical work. Every chapter discusses both a fundamental philosophical position on the social sciences and a relevant case study from the social sciences.

Researchers organize the world around them into categories. Their choices in categorizing a phenomenon of interest are never only determined by the area under consideration: there is flexibility in (amongst other things) scientists’ choices of category boundaries, number of categories, and rules for determining category membership. This article treats categorization choices as conventions, and accordingly, develops an information theoretic measure from the philosophy of social conventions to track how much flexibility a researcher has in some given area of categorization. Following work by Brian Skyrms and Cailin O’Connor, the article uses Shannon entropy to define the level of arbitrariness of categorization. It is then argued that two interpretations of arbitrariness are salient for philosophers: a descriptive interpretation, which delivers the amount of consensus in any given area as well as changes in this amount over time; and a normative interpretation, which systematizes philosophical questions on pluralism, i.e., the intuitive notion that some areas of inquiry benefit from there being a variety of ways of categorizing. By discussing categorization as a form of clustering, this article shows that the normative interpretation is related to, but not reducible to, the question of whether the phenomenon under study is a kind. Contrasting the two interpretations, this article shows the importance of distinguishing the degree of consensus on the one hand from the normative degree of arbitrariness on the other. This distinction further motivates quantifying conventionality, as it calls into question the idea that consensus in categorization is always a sign of scientific progress.

This article responds to Yafeng Shan and Jon Williamson’s 2022 volume Evidential Pluralism in the Social Sciences, focusing on the applicability of Evidential Pluralism in the field of case study research through the use of examples from political science. The article argues that Shan and Williamson’s guidance on (a) what evidence one needs to establish causation in singular case studies and (b) what one may conclude at the population level once one has done so is lacking in some important respects. Therefore, this article will speak to the wider community of social scientists and methodologists who are sympathetic to using a plurality of evidence in case study research, but who still want guidance of how to do so in a philosophically grounded manner.

Limits of the Numerical calls for the (re)contextualization of the numerical in the social domain and emphasizes that using quantitative data has epistemic and practical/moral considerations that may not align. In this review essay, we evaluate these claims using a case study, viz. the personalized, clinical experience sampling method (ESM) in mental health care. This case study (1) nuances claims made in Limits of the Numerical regarding the generality and non-contextuality of numerical data, and (2) highlights two additional dimensions to the numerical that have been underexposed in the book (i.e., reactivity and Ballung concepts).

This article uses the interventionist theory of causation, a counterfactual theory taken from philosophy of science, to strengthen causal analysis in process tracing research. Causal claims from process tracing are re-expressed in terms of so-called hypothetical interventions, and concrete evidential tests are proposed which are shown to corroborate process tracing claims. In particular, three steps are prescribed for an interventionist investigation, and each step in turn is shown to make the causal analysis more robust, amongst others by disambiguating causal claims and clarifying or strengthening the existing methodological advice on counterfactual analysis. The article's claims are then illustrated using a concrete example, Haggard and Kaufman's analysis of the Argentinian transition to democracy. It is shown that interventionism could have strengthened the authors’ conclusions. The article concludes with a short Bayesian analysis of its key methodological proposals.

As a result of being measured, individuals sometimes alter their behavior and attitudes to such extent that subsequent measurement results are affected. This ‘reactivity’ to measurement problematizes prediction and explanation, but some reactivity is nevertheless legitimate. Using the example of the measurement of race in the US Census, this article demonstrates that some forms of reactivity do not affect the accuracy of research. The article argues that legitimacy of reactivity depends on the metaphysical status of the phenomenon being measured. It is argued that reactivity’s legitimacy is affected by the arbitrariness of a measure and the voluntariness of measurement results.

Evidential pluralists, like Federica Russo and Jon Williamson, argue that causal claims should be corroborated by establishing both the existence of a suitable correlation and a suitable mechanism complex. At first glance, this fits well with mixed method research in the social sciences, which often involves a pluralist combination of statistical and mechanistic evidence. However, statistical evidence concerns a population of cases, while mechanistic evidence is found in individual case studies. How should researchers combine such general statistical evidence and specific mechanistic evidence? This article discusses a very recent answer to this question, ‘multi-method large-N qualitative analysis’ or multi-method LNQA, popular in political science and international relations studies of rare events like democratic transitions and cease-fire agreements. Multi-method LNQA combines a comprehensive study of all relevant event cases with statistical analysis, in an attempt to solve the issues of generalization faced by other types of qualitative research, such as selection bias and lack of representativeness. I will argue that the kind of general causal claim that multi-method LNQA is after, however, is crucially different from the average treatment effect found in statistical analysis and can in fact only be supported with mechanistic evidence. I conclude from this that mixed method research, and thereby evidential pluralism, may be inappropriate in this context.

If a human subject knows they are being measured, this knowledge may affect their attitudes and behaviour to such an extent that it affects the measurement results as well. This broad range of effects is shared under the term ‘reactivity’. Although reactivity is often seen by methodologists as a problem to overcome, in this paper I argue that some quite extreme reactive changes may be legitimate, as long as we are measuring phenomena that are not simple biological regularities. Legitimate reactivity is reactivity which does not undermine the accuracy of a measure; I show that if such reactivity were corrected for, this would unjustifiably ignore the authority of the research subject. Applying this argument to the measurement of depression, I show that under the most commonly accepted models of depression there is room for legitimate reactivity. In the first part of the paper, I provide an inventory of the different types of reactivity that exist in the literature, as well as the different types of phenomena that one could measure. In the second part, I apply my argument to the measurement of depression with the PHQ-9 survey. I argue that depending on what kind of phenomenon we consider depression to be, we will accept different kinds of reactivity. I show that both under the harmful dysfunction model and the practical kinds model, certain reactive changes in measuring depression are best seen as legitimate recharacterizations of the underlying phenomenon, and define what legitimate means in this context. I conclude that in both models, biological aspects constrain characterization, but the models are not so strict that only one concept is acceptable, leaving room for reactivity.

Evidential pluralism has been used to justify mixed-method research in political science. The combination of methodologies within (qualitative) case study analysis, however, has not received as much attention. This article applies the theory of evidential pluralism to causal inference in the case study method process tracing. I argue that different methodologies for process tracing commit to distinct fundamental theories of causation. I show that, problematically, one methodology may not recognize as genuine knowledge the fundamental claims of the other. By evaluating the epistemic reliability of these fundamental claims, we can find a way out of such conflicts and rescue pluralism.

Philosophers of social science have emphasized mechanistic approaches to causal inquiry for some time now, showing why focusing on the mechanisms behind correlations is preferable to focusing on correlations alone (cf. Johnson 2006, Little 1991, Reiss 2007, 2009, Steel 2004, see also King, Keohane, and Verba 1994 for an example of purely correlational research). In Process Tracing: from Metaphor to Analytic Tool, political scientists Andrew Bennett and Jeffrey Checkel present a concrete method for finding evidence of causal mechanisms, process tracing. Process tracers give evidence for causal relations in terms of the observable implications of the underlying causal mechanisms through which a putative cause affects some effect of interest. Such observable implications often take the form of a chain of events, or process, which connects cause and effect. Though Process Tracing contains applications of mechanistic reasoning unfamiliar to philosophers, and as such will be of interest to those working in the mechanist tradition, Bennett and Checkel’s own discussion of the philosophical foundations of process tracing is limited. The reader will have to delve deep into the volume’s contributed chapters for links between the literature on causation and this new method.

The issue of causal comparability in the social sciences underlies matters of both generalization and extrapolation. After critiquing two existing interpretations of comparability, due to Hitchcock and Hausman, I propose a distinction between ontological and epistemic comparability. While the former refers to whether two cases are actually comparable, the latter respects that in cases of incomplete information, we need to rely on whatever evidence we have of comparability. I argue, using a political science case study, that in those cases of imperfect information, an epistemic homogeneity criterion can be an adequate justification for generalization.

In this paper, I investigate the study of causal mechanisms in the social sciences. I argue that unless one adopts a clear notion of causation, such as Woodward's manipulability theory of causation, one does not find evidence for causal claims. I show that adopting Woodward’s theory entails that a researcher must take into account both the observable implications of the mechanisms, and possible interventions on those mechanisms. In a backlash against the pervasiveness of statistical methods, in the last decade or so social scientists have become more focused on finding the causal mechanisms behind observed correlations. To provide evidence for such causal claims, researchers often rely on process-tracing, a method which involves contrasting the observable implications of several alternative mechanisms. Although process-tracing has gained prominence as a technique, researchers do not share a fundamental framework of what causation is. I will show what process-tracers’ reasoning would look like if they were to commit to Woodward’s manipulability theory. This theory tells us that any successful explanation of an effect must refer to causal factors that could be manipulated to change the phenomenon under study. Specifically, a variable X is a cause of another variable Y if there exists some ‘intervention variable’ I. I is an intervention variable if we can use I to change X, which will then in turn change Y without any interference of other variables linked to Y. In other words, using I we will be able to ascertain that it was X that made the change in Y happen. I will argue that if a process-tracing researcher were to adopt the manipulability notion of causation, she would not only have to find observable implications of the alternative mechanisms under study, but also information regarding an intervention variable for each link of the causal chain.

In a backlash against the prevalence of statistical methods, recently social scientists have focused more on studying causal mechanisms. They increasingly rely on a technique called process-tracing, which involves contrasting the observable implications of several alternative mechanisms. Problematically, process-tracers do not commit to a fundamental notion of causation, and therefore arguably they cannot discern between mere correlation between the links of their purported mechanisms and genuine causation. In this paper, I argue that committing to Woodward's interventionist notion of causation would solve this problem: process-tracers should take into account evidence for possible interventions on the mechanisms they study.

In educational research, the ‘paradigm wars’—vehement debates between proponents of qualitative approaches and proponents of quantitative approaches—are far from over (Bryman 2006). The development of British educational research demonstrates this nicely. Influenced by policy makers’ demands for prediction and control, the current research climate is characterised by a mistrust of qualitative methods, particularly in regard to their potential value-ladenness and lack of rigour (Hodkinson 2008).British education sociologist and methodologist Martyn Hammersley, one of the authors of Challenging the Qualitative–Quantitative Divide: Explorations in Case-Focused Causal Analysis, objects to the dichotomy between qualitative and quantitative methods that the paradigm wars are based on. He argues that the division is artificial and may even be damaging: for instance, it could discourage researchers of one approach from taking into account arguments produced by researchers of a different style ..

In this thesis I investigate causal inquiry in the social sciences, drawing on examples from various disciplines and in particular from conflict studies. In a backlash against the pervasiveness of statistical methods, in the last decade certain social scientists have focused on finding the causal mechanisms behind observed correlations. To provide evidence for such mechanisms, researchers increasingly rely on ‘process tracing’, a method which attempts to give evidence for causal relations by specifying the chain of events connecting a putative cause and effect of interest. I will ask whether the causal claims process tracers make are defensible, and where they are not defensible I will ask how we can improve the method. Throughout these investigations, I show that the conclusions of process tracing are constrained both by the causal structure ofthe social world and by social scientists’ aims and values. My central argument is this: all instances of social phenomena have causally relevant differences, which implies that any research design that requires some comparison between cases is limited by how we systematize these phenomena. Moreover, such research cannot rely on stable regularities. Nevertheless, to forego causal conclusions altogether is not the right response to these limitations; by carefully outlining our epistemic assumptions we can make progress in causal inquiry. While I use philosophical theories of causation to comment on the feasibility of a social scientific method, I also do the reverse: by investigating a popular contemporary method in the social sciences, I show to what extent our philosophical theories of causation are workable in practice. Thus, this thesis is both a methodological and a philosophical work. Every chapter discusses both a fundamental philosophical position on the social sciences and a relevant case study from the social sciences.