Daedo Jun

Human evaluation lies at the center of how AI systems are built and trusted. Benchmarks are constructed from human labels; reinforcement learning from human feedback (RLHF) treats aggregated human judgment as a proxy for quality; safety assessments rely on human raters to identify harmful outputs. Underlying all of these systems is a shared implicit assumption: that human judgment, when consistently applied, approximates objective quality. This study challenges that assumption — not through theoretical argument, but through data. Analyzing 1,500 sentence-level evaluations drawn from 300 GPT-4o-generated text samples, we find that five independent raters diverged by 22.1 percentage points in their grounding assessments. Yet the disagreement was not random. Across all raters and all conditions, a strict monotonic pattern held without exception: R0 < R1 < R2. As structural depth increased, inter-rater variance narrowed. We term this phenomenon Structured Divergence. Disagreement is not noise. It is the systematic expression of different interpretive standards encountering structurally graded output. Human beings do not read the same sentence and see the same world. And that difference is not a defect to be corrected — it is the condition of meaning itself. The conclusion is straightforward: human evaluation is not broken. It has been misread.

This paper reconceptualizes hallucination in large language models (LLMs) as a form of semantic reliability failure, in which internal meaning structures lose coherence across depth and context. Rather than treating hallucination as an isolated factual error, we frame it as a disruption of semantic stability arising from misalignment among intention, evidence, and contextual resonance. To address this, we introduce the Layer-Knot Framework (LKF)—a structural mechanism that anchors meaning through inter-layer semantic knots, maintaining topological coherence within the model’s representational space. To operationalize semantic reliability, the study proposes a triadic evaluation system consisting of Hallucination Rate (HR), Groundedness Rate (GR), and Coherence Rate (CR). These indicators jointly capture the dynamic balance between factual accuracy, evidential consistency, and generative diversity. Experiments conducted on TruthfulQA and 2,000 domain-specific prompts demonstrate that LKF reduces hallucination by 50%, increases groundedness by 12%, and preserves creative variance within ±3%. The findings show that reliability and creativity are not competing objectives but can coexist when a model’s internal semantic topology is structurally aligned. This reframes hallucination suppression as a process of semantic alignment, rather than a constraint on generative capacity. The proposed HR/GR/CR metrics provide a unified methodological basis for evaluating AI trustworthiness, while LKF offers an architectural pathway toward structurally resilient, self-consistent, and ethically aligned LLMs. Overall, the study establishes a foundational link between linguistic integrity and ethical reliability, and it outlines future directions for developing LLMs capable of self-monitoring their own semantic truthfulness as an emergent form of conscious alignment.