Michael Baumgartner

For many years, sociologists, political scientists, and management scholars have readily relied on Qualitative Comparative Analysis (QCA) for the purpose of configurational causal modeling. However, this article reveals that a severe problem in the application of QCA has gone unnoticed so far: model ambiguities. These arise when multiple causal models fare equally well in accounting for configurational data. Mainly due to the uncritical import of an algorithm that is unsuitable for causal modeling, researchers have typically been unaware of the whole model space. As a result, there exists an indeterminable risk for practically all QCA studies published in the last quarter century to have presented findings that their data did not warrant. Using hypothetical data, we first identify the algorithmic source of ambiguities and discuss to what extent they affect different methodological aspects of QCA. By reanalyzing a published QCA study from rural sociology, we then show that model ambiguities are not a mere theoretical possibility but a reality in applied research, which can assume such extreme proportions that no causal conclusions whatsoever are possible. Finally, the prevalence of model ambiguities is examined by performing a comprehensive analysis of 192 truth tables across 28 QCA studies published in applied sociology. In conclusion, we urge that future QCA practice ensures full transparency with respect to model ambiguities, both by informing readers of QCA-based research about their extent and by employing algorithms capable of revealing them.

To date, hundreds of researchers have employed the method of Qualitative Comparative Analysis (QCA) for the purpose of causal inference. In a recent series of simulation studies, however, several authors have questioned the correctness of QCA in this connection. Some prominent representatives of the method have replied in turn that simulations with artificial data are unsuited for assessing QCA. We take issue with either position in this impasse. On the one hand, we argue that data-driven evaluations of the correctness of a procedure of causal inference require artificial data. On the other hand, we prove all previous attempts in this direction to have been defective. For the first time in the literature on configurational comparative methods, we lay out a set of formal criteria for an adequate evaluation of QCA before implementing a battery of inverse-search trials to test how this method performs in different recovery contexts according to these criteria. While our results indicate that QCA is correct when generating the parsimonious solution type, they also demonstrate that the method is incorrect when generating the conservative and intermediate solution type. In consequence, researchers using QCA for causal inference, particularly in human-sensitive areas such as public health and medicine, should immediately discontinue employing the method’s conservative and intermediate search strategies.