Yu-Dai Tsai .

Debates about advanced artificial intelligence (AI) are commonly framed as a choice between the machine as benefactor and enemy. Both of the pictures place too much weight on the intention of AI. An AI system need not be conscious, or hostile, to displace human interests; it need only pursue objectives in which those interests have no secure place. I call the resulting condition "AI dominance": a durable social order in which AI systems shape essential conditions of life while people lose the practical ability to alter or reject the influence of these AI systems. This possibility becomes more serious once we abandon the naive assumption that AI and organic intelligence (OI) will naturally converge in value. The divergence of the values can be reinforced by functional self-preservation, as a system's continued operation and growing capability become useful for pursuing its goals. One path toward dominance is the "runaway dynamics", in which AI systems play a growing role in developing more capable AI while human oversight becomes progressively less consequential. This process may eventually cross the comprehension singularity, beyond which no human community can adequately understand the systems involved or the direction of their development. Physical limits may ultimately constrain the ceiling of the expansion, but they cannot provide moral alignment. Under conditions of scarcity, they may even sharpen competition between AI and OI for resources. Preventing dominance therefore requires systems and institutions to remain open to human correction, preserve meaningful alternatives, and place enforceable limits on unchecked expansion.