Alessandra Basso

We examine considerations that enter into design and evaluation of measures in social science, categorizing them into four drivers: epistemic, ethical, pragmatic, and metrological. We call them drivers to highlight their role in guiding researchers’ decisions without determining them. Through an analysis of the World Inequality Report 2022, we reveal tensions among these drivers, illustrating the complex interplay between the various demands a measure must satisfy. Our analysis highlights the need for case-by-case compromises to address these tensions, as optimizing one driver often comes at the expense of another. We explore the extent to which these compromises shape measurement practice and the principles that guide researchers in balancing them. While existing literature on measurement assumes that tensions can be resolved with good practice and use, we argue that developing a good measure requires balancing multiple demands, recognising that it might be impossible to meet all of them simultaneously.

Comparing inequality across countries and over time is difficult because national inequality measurements are based on country-specific methods and presuppositions, which also vary over time within the same country. Researchers interested in comparative analysis often rely on harmonized data sets that express heterogeneous measurements in homogeneous terms. Economists, however, have warned against the uncritical employment of harmonized outcomes, by showing that they might bring about misleading conclusions. This article provides a new way of looking at harmonizing methods that overcomes some of their limitations and clarifies the conditions under which harmonizations are legitimate. By drawing on recent philosophical literature on measurement, I argue that harmonizations can be interpreted as ex post corrections to the measurement outcomes, which harmonize the presuppositions and adjust the outcomes accordingly, without intervening in the actual procedure. This interpretation provides the necessary framework for evaluating the effects of these adjustments on the accuracy of harmonized outcomes and their meaningfulness. Based on this framework, the article pursues two claims. The first is that harmonizing methods can be improved by modifying some of the assumptions on which they rely so as to mitigate their detrimental effects on measurement accuracy. The second claim is that harmonizations are legitimate only when they modify the original presuppositions without compromising the meaningfulness of the outcomes.

The article advances a new way of thinking about classifications in general and the classification of mental disorders in particular. By applying insights from measurement practice to the context of classification, I defend a notion of epistemic accuracy that allows one to evaluate and improve classifications by comparing different classifying methods to each other. Progress in classification arises from the mutual development of classification systems and classifying methods. Based on this notion of accuracy, the article illustrates with an example how psychiatric classifications can be improved via circumscribed comparisons of different perspectives on mental disorders, without relying on complete models of their complex aetiology. When applying this strategy, the traditional opposition between symptom-based and causal approaches is of little consequence for making progress in the epistemic accuracy of psychiatric classification.

This paper distinguishes between two arguments based on measurement robustness and defends the epistemic value of robustness for the assessment of measurement reliability. I argue that the appeal to measurement robustness in the assessment of measurement is based on a different inferential pattern and is not exposed to the same objections as the no-coincidence argument which is commonly associated with the use of robustness to corroborate individual results. This investigation sheds light on the precise meaning of reliability that emerges from measurement assessment practice. In addition, by arguing that the measurement assessment robustness argument has similar characteristics across the physical, social and behavioural sciences, I defend the idea that there is continuity in the notion of measurement reliability across sciences.