Kun Yuan

AI safety, risk, and governance are becoming global agenda items, but the intensification of governance activity does not automatically clarify the object of risk. When different fields say that they seek to protect “humanity,” they implicitly point to preferences, lives, social order, or species continuity—concepts that are not mutually compatible. As a result, risk identification continually slides across dimensions. This paper is an exercise in conceptual engineering concerning the risk-bearing subject of AI-related existential risk to humanity. It also has significance for social epistemology, the philosophy of technology, and AI risk governance. This paper preserves the baseline of existing frameworks that understand “humanity” as Homo sapiens, but proposes a paradigm extension: the risk-bearing subject is always humanity as an embodied species; what this paper extends is not the risk-bearing subject, but the ways in which one and the same subject may suffer existential harm. The paper distinguishes two basic layers of possible harm—the biological-survival layer and the civilizational-conditions layer—and, within the latter, identifies the philosophical kernel, namely the category-calibration mechanism of foundational categories such as truth, evidence, responsibility, subjecthood, value, and meaning. AI may, through deprivation by substitution or mediated reconfiguration, weaken humans’ substantive participation in the functional systems of civilization, or cause foundational categories to lose stable calibration. Risk to the realization of subjecthood at the civilizational-conditions layer enters the scope of existential risk only when it acts upon foundational civilizational conditions, has diffusion capacity across domains and civilizational interfaces, may cause irreversible or difficult-to-reverse structural damage, and causes these harms to converge into overall, intergenerational damage to humanity’s capacity as a future-oriented subject. What this paper provides is not a complete taxonomy of risks or a governance scheme, but the ontological coordinate system needed for AI risk identification. The central point is not to add civilization, institutions, or the philosophical kernel as new risk-bearing subjects, but to show how the same human subject may suffer existential harm through damage to civilizational conditions.

This paper is a work of conceptual engineering on the risk-bearing subject of AI-related existential risk to humanity. It also has significance for social epistemology, philosophy of technology, and AI risk governance. Existing discussions usually understand “humanity” as the biological species *Homo sapiens*, and accordingly define the risk in terms of species extinction, permanent incapacitation, or future deprivation. This paper preserves the baseline status of that framework, but proposes a paradigm extension: while the risk-bearing subject remains humanity as an embodied species, the possible modes of existential harm to that subject should be extended from destruction or permanent incapacitation at the single level of biological survival to the irreversible or difficult-to-reverse weakening of the foundational conditions under which humanity realizes subjecthood within civilization. The central question of this paper is: when we say that AI may threaten “humanity,” what exactly is the “humanity” at risk? The paper distinguishes two basic layers of possible harm to humanity as the risk-bearing subject: the biological-survival layer and the civilizational-conditions layer. The former concerns the survival of humanity as an embodied species; the latter concerns humanity’s capacity, within an intergenerational common world, to know what is true, form judgment, bear responsibility, participate in institutions, and create meaning. Within the civilizational-conditions layer, the paper further identifies the category-calibration mechanism: the mechanism through which foundational categories such as truth, evidence, responsibility, subjecthood, value, and meaning are clarified, maintained, and revised. The philosophical kernel is not an independent entity, but the category-calibration mechanism internal to civilizational conditions that maintains the operability of foundational categories. AI may weaken humans’ substantive participation in the functional systems of civilization, or destabilize the calibration of foundational categories, through **deprivation by substitution** or **mediated reconfiguration**. The expanded definition proposed in this paper is as follows: AI-related existential risk to humanity refers to the risk, arising from the development, deployment, integration, or loss of control of AI systems, that AI may irreversibly destroy the human species; or render humanity permanently incapacitated, or otherwise permanently foreclose humanity’s future; or undermine—irreversibly or in ways that are difficult to reverse—the foundational conditions under which humanity, as an embodied species, realizes its subjecthood within civilization. Within this definition, risks to the realization of human subjecthood at the civilizational-conditions layer are constrained by **four thresholds**: they must act upon foundational civilizational conditions, possess cross-domain diffusion capacity, be capable of causing irreversible or difficult-to-reverse structural damage, and cause the first three forms of damage to converge into overall damage to humanity’s capacity as a future-oriented subject. What this paper provides is not a complete risk taxonomy, nor a technical control scheme or policy checklist, but an **ontological coordinate system** needed at the level of risk identification in AI safety, AI risk analysis, and AI governance. Before discussing specific risk types, governance pathways, or technical solutions, we must first clarify the risk-bearing subject called “humanity” and the possible modes of harm to it.

This paper develops an ontological and conceptual-engineering framework for understanding AI-related human existential risk. As AI systems become more capable, more general-purpose, more agentic, and more deeply embedded in human knowledge production, institutional processes, judgment formation, and technological development, AI risk can no longer be understood merely as a problem of model performance, product safety, or engineering governance. In frontier technical contexts, AI is increasingly used to assist code generation, model evaluation, research automation, and the development of AI systems themselves. In public-ethical contexts, AI is also increasingly understood as posing categorical challenges to human dignity, responsibility, freedom, the common good, and the self-understanding of human societies. These developments reopen the question of human existential risk. Yet in many discussions of AI risk, a more basic question remains insufficiently clarified: when we say that AI may pose an existential risk to humanity, what exactly is the “humanity” that is at risk? This paper argues that AI-related human existential risk should not be assessed solely in terms of whether *Homo sapiens*, as a biological species, is physically destroyed, permanently disempowered, or deprived of its future. Human beings certainly exist as a biological species, but humanity also exists as a civilizational subject: an intergenerational common world constituted by language, knowledge, institutions, responsibility, truth, agency, value, and meaning. If AI does not directly destroy the human species, but irreversibly weakens the basic conditions under which humanity can continue to exist as a civilizational subject, then this form of risk should also be included within an expanded understanding of AI-related human existential risk. To develop this claim, this paper distinguishes three nested layers: the human species as the biological substrate of civilization; human civilization as the intergenerational common operating environment formed by the species; and philosophy as the reflective kernel through which civilization defines and revises its foundational categories. The human species provides the biological substrate of civilization. Human civilization enables the species to become a collective subject capable of knowledge, judgment, responsibility, transmission, and meaning-making. The philosophical kernel supplies foundational categories—such as truth, evidence, responsibility, agency, value, and meaning—that are presupposed by civilizational systems such as science, law, politics, education, economy, culture, and the arts. This paper uses a functional hardware–kernel–distribution analogy to clarify the relation among these layers. The human species may be compared to hardware; philosophy to a kernel; and human civilization to a distribution or complete operating environment running on a biological substrate and structured by a philosophical kernel. The analogy is neither literal nor historical. Its purpose is to illuminate the layered structure of human existence: hardware may remain intact while the system fails to operate well; a distribution may still boot while the kernel is damaged; the application layer may continue to flourish while foundational categories deteriorate. It concludes by proposing an expanded definition: AI-related human existential risk arises not only when AI threatens the biological survival of humanity, but also when AI irreversibly weakens the civilizational conditions under which humanity can continue to exist as a subject of truth, responsibility, agency, and meaning. This definition does not attempt to classify all AI-driven social change as existential risk. Rather, it provides a conceptual basis for further analysis of the major dimensions of human existential risk in the age of AI and AGI.