Tad Zawidzki

Recent advances in large language models (LLMs) have sparked intense debates concerning their cognitive abilities. Despite their impressive achievements, LLMs still fall short of human performance in social cognition and metacognition. Here, we argue that many of these limitations share a common and often neglected root. Human social cognition and metacognition both crucially involve normative cognition. Roughly, this consists in our ability to distinguish between correct and incorrect thoughts and behaviors, and to regulate others and ourselves in light of such attitudes. Indeed, attributing beliefs or intentions to others serves not only to predict and explain behavior, but also to justify it with respect to social and rational norms. Similarly, in metacognition, we aim not only to track our inner cognitive processes, but also to regulate them according to these same norms. The normative attitudes which are necessary for these uniquely human socio-cognitive and metacognitive abilities have not yet been implemented in LLMs, and this, we argue, is the root of many of their current limitations in these domains. Moreover, this is not due to some inherent technological challenge. Rather, proper normative attitudes have not been implemented in LLMs because of ethical and political concerns: this would entail conferring on them substantial ethical and political status in our communities. Making artificial agents more human-like in their social cognition and metacognition may require providing them with capacities entitling them to something akin to the normative status of persons, which is in tension with the goal of using them as tools.