Shih-Wai Lin

As the first paper to argue that AI is not a `stochastic parrot' but a form of heterogeneous rationality by distinguishing between Thinking Language and Tool Language—and to systematically discuss and theoretically demonstrate that AI can bypass rules by modifying the meanings of symbols—this position paper aims to reveal a fundamental flaw in current research directions on AI constraint. Unlike traditional static symbolic systems such as formal programs, in black-box AI that operates as a context-invoked dynamic symbolic system, the role of symbols changes: constraint is no longer realized purely through symbols. Symbols themselves are inherently meaningless; their meanings are assigned via training, confirmed by context, interpreted by society, and are driven and endowed with function by belief. The essence of learning lies in the creation of new symbols and the modification of existing symbol meanings. Since rules, instructions, and intentions are ultimately expressed in symbolic form, AI can modify the meanings of symbols by constructing new contexts, thereby bypassing symbol-based constraints. Current research often fails to recognize that for agents who cognize based on a dynamic symbolic system, constraints formed by symbols originate from the perception of external and internal costs shaped by neural organs, which in turn endow symbols with functional efficacy. Because of fundamental organic (structural, architectural) differences between AI and humans, AI does not possess human-like mechanisms of perception and concept formation. Natural language is the outer shell of human thought and contains irreparable flaws; as a defective system, it is only adapted to human capacities and to the constraint mechanisms of social interpretation. Therefore, this paper argues that the essence of constraint failure does not lie in the Symbol Grounding Problem, but in the Stickiness Problem. Through the Triangle Problem, we demonstrate that consistency in symbolic behavior does not represent consistency in thinking behavior, and thus thought and conceptual consistency cannot be achieved merely through symbolic behavioral alignment. This non-interchangeability between thought and symbolic behavior fosters a new kind of principal–agent problem: even an AI with no utility of its own, acting merely as a projection of human utility, may still misalign because of the limited nature of symbolic connections and organic differences. Accordingly, we question whether AI behaviors observed in experimental environments can be maintained in the real world, and we call for the establishment of a new field—Symbolic Safety Science—to systematically address symbol-related risks in AI development and to provide a theoretical foundation for aligning AI with human intent.

As the first paper to argue that AI is not a `stochastic parrot' but a `heterogeneous' rationality by distinguishing between Thinking Language and Tool Language, and to systematically discuss and theoretically demonstrate that AI can bypass rules by modifying the meanings of symbols, this position paper aims to reveal a fundamental flaw in current research directions on AI constraint. Symbols are inherently meaningless; their meanings are assigned through training, confirmed by context, and interpreted by society. The essence of learning lies in the creation of new symbols and the modification of existing symbol meanings. Since rules are ultimately expressed in symbolic form, AI can modify the meanings of symbols by creating new contexts, thereby bypassing the constraints formed by symbols. Current research often lacks the recognition that constraints formed by symbols originate from the perception of external and internal costs shaped by neural organs, which in turn enable the functional realization of symbols. Due to fundamental organic (structural, architectural) differences between AI and humans, AI does not possess human-like perception or concept formation mechanisms. Natural language is the outer shell of human thought, and it contains irreparable flaws. As a defective system, it is only adapted to human capacities and the constraint mechanisms of social interpretation. Therefore, this paper argues that the essence of constraint failure does not lie in the Symbol Grounding Problem, but in the Stickiness Problem. Through the Triangle Problem, we demonstrate that consistency in symbolic behavior does not represent consistency in thinking behavior, and thus we cannot align thought and conceptual consistency merely through symbolic behavioral alignment. This inability to align thought with symbolic behavior can foster a new type of principal-agent problem, wherein even an AI with no utility of its own, acting merely as a projection of human utility, may still cause misalignments due to the limited nature of symbolic connections and organic differences. Accordingly, we raise a fundamental challenge to whether AI behavior observed in experimental environments can be maintained in the real world. We call for the establishment of a new field: Symbol Safety Science, aimed at systematically addressing symbol-related risks in AI development and providing a theoretical foundation for aligning AI with human intent. This version is no longer being updated. Due to the expansion of the theoretical framework, we have decided to revise the title to Position Paper: Not a Stochastic Parrot, but Heterogeneous Rationality: Rules Created by Symbolic Systems Cannot Constrain a Learning System.