Chris Santos-Lang

This paper contributes a new way to evaluate AI. Much as one might evaluate a machine in terms of its performance at chess, this approach involves evaluating a machine in terms of its performance at a game called “MAD Chairs.” At the time of writing, evaluation with this game exposed opportunities to improve Claude, Gemini, ChatGPT, Qwen and DeepSeek. Furthermore, this paper sets a stage for future innovation in game theory and AI safety by providing an example of success with non-standard approaches to each: studying a game beyond the scope of previous game theoretic tools and mitigating a serious AI safety risk in a way that requires neither determination of values nor their enforcement.

Social reform has historically been proposed and led by humans, but we may wonder if some future social reforms could be proposed or led by AI. If negligence is blameworthy, then AI might even deserve blame if it never serves as our moral and social leader (or we might deserve blame for never producing AI that should serve in that way). As a concrete and measurable example of where AI could lead us to treat each other better, this paper offers a game-theoretic examination of a common resource-division problem. Study 1 reports recent results of behavioural economics experiments with this multi-player game. The human behaviour observed in the study is so far from optimal as to imply that ceding some autonomy to AI-or at least some leadership to AI-might be in our best interest. Rather than mitigate the social harms associated with caste systems (such as racism, sexism, etc), the anonymity of the online game setting appears to simply reallocate harm less systematically. Thus, to actually cure some of our most pressing modern social problems may even require guidance by some form of AI. Standard open-source oTree software to reproduce these results is made available on GitHub. Study 2 involves philosophical thought experiments in the style of Nozick's "Experience machine". To facilitate discussion about when suboptimality might qualify as "safe" generally, an evidence-based leadership-sharing framework aligning with the Scientist AI approach of LawZero and Yoshua Bengio is conceptualized to ask when one might preserve the opportunity to make mistakes (i.e., to not use the hypothetically perfect machine). Would we extend the same privilege to AI? While Study 1 offers evidence to support the Scientist AI approach for some contexts, we believe Study 2 justifies urgency for a deeper discussion about the pursuit of optimality and its consequences for our own autonomy.

One can relinquish one’s autonomy, can allow something else—an AI, evidence, or coinflip—to govern one’s behavior. For example, a chess novice who trusts Stockfish to pick moves relinquishes autonomy compared to a player who follows only advice they understand. We conducted AI tournaments with basic social games to predict consequences of adding advised players (or robots) to populations who divide scarce resources. For example, what happens to traffic when some vehicles become automated? What happens to allocation of jobs in a tough economy, to prime real estate, and to representation in a democracy that cannot practically consider all perspectives? We measured four prices: The damage the advice would do to individuals who ignore it, the profit that others would forego if denied advice, and the profit lost to each by allowing autonomy to be retained. Open source code for these tournaments in oTree format is available on github.

Wallach and Allen’s seminal book, Moral Machines: Teaching Robots Right from Wrong, categorized theories of machine ethics by the types of algorithms each employs (e.g., top-down vs. bottom-up), ultimately concluding that a hybrid approach would be necessary. Humans are hybrids individually: our brains are wired to adapt our evaluative approach to our circumstances. For example, stressors can inhibit the action of oxytocin in the brain, thus forcing a nurse who usually acts from subjective empathy to defer to objective rules instead. In contrast, ecosystem approaches to ethics promote hybridization across, rather than within, individuals; the nurse being empowered to specialize in personalized care because other workers specialize in standardization, and profitability. Various philosophers have argued, or laid the framework to argue, that such specialization can be advantageous to teams and societies. Rather than mass-produce identical machines to emulate the best individual human, perhaps we should build diverse teams of machines to emulate the best human teams.

The ecosystem approach to computer system development is similar to management of biodiversity. Instead of modeling machines after a successful individual, it models machines after successful teams. It includes measuring the evaluative diversity of human teams (i.e. the disparity in ways members conduct the evaluative aspect of decision-making), adding similarly diverse machines to those teams, and monitoring the impact on evaluative balance. This article reviews new research relevant to this approach, especially the validation of a survey instrument for measuring computational evaluative differences in humans (the GRINSQ). The research confirms the existence of all four known machine types among humans.

This essay seeks to advance a discussion to meet the needs of designers of technologies–including of institutions–which are meant to help users accurately answer questions about reality, including questions about nature and morality. Specifically, it helps clarify the list of requirements that designs would have to satisfy in order to provide reasonable expectation that the technology would converge on truth. The design of the procedures of the Belleville Research Ethics Committee (BREC) are offered as a practical example of how such a list of requirements might inform design, and this essay aims to help other designers reason about potential improvement to the BREC procedures.

Science fiction about intelligent aliens has long imagined a science of sociology with typologies that apply universally, much as the periodic table of the elements applies to atoms on all planets. The GRIN model purports to offer such a universal typology. This study offers the first instrument to measure its manifestation in humans: the Gadfly-Relational-Institutional-Negotiator Self-Quiz (GRINSQ). It reports evidence of the GRINSQ's reliability, as well as its structural, content, convergent and pragmatic validity, including relationships to the Moral Judgment Test (MJT), Moral Foundations Questionnaire (MFQ), and Big Five personality traits. The evidence supports the hypotheses that humans specialize by GRIN-type, and that this specialization relates to differences in personality, morality, political orientation, career, religion, family type, and identification with crime.