Marko Tesic

As AI is integrated into the workplace, organisations increasingly face allocation decisions between human and machine workers. These decisions are increasingly made or assisted by algorithms, creating a Reverse Turing Test dynamic wherein the machine is now the judge. In addition, human and machine workers may ``compete'' for a given task, reproducing aspects of adversarial games. This raises new methodological questions about assessing task suitability between humans and machines. The criteria often used to assess people (e.g., education, experience, references) cannot feasibly scale to AI systems; conversely, AI evaluation methods (benchmarks, red teaming, leaderboards) cannot be easily applied to human workers or yield comparable metrics. In this position paper, we argue that suitability evaluations for task-assignment should be profile-driven -- that is, based on assessments that infer latent constructs such as capabilities and propensities from observed performance. This approach places humans and AI systems on shared scales, supporting comparisons that are predictive of novel-task performance, explanatory of why agents succeed or fail, and auditable. We outline the core features of this approach, discuss its practical implications, and compare it with alternative frameworks for human-machine workplace allocation.

In their 2010 paper, Dizadji-Bahmani, Frigg, and Hartmann argue that the generalized version of the Nagel–Schaffner model that they have developed is the right one for intertheoretic reduction, i.e. the kind of reduction that involves theories with largely overlapping domains of application. Drawing on the GNS, DFH presented a Bayesian analysis of the confirmatory relation between the reducing theory and the reduced theory and argued that, post-reduction, evidence confirming the reducing theory also confirms the reduced theory and evidence confirming the reduced theory also confirms the reducing theory, which meets the expectations one has about theories with largely overlapping domains. In this paper, I argue that the Bayesian analysis presented by DFH faces difficulties. In particular, I argue that the conditional probabilities that DFH introduce to model the bridge law entail consequences that run against the GNS. However, I also argue that, given slight modifications of the analysis that are in agreement with the GNS, one is able to account for these difficulties and, moreover, one is able to more rigorously analyse the confirmatory relation between the reducing and the reduced theory.