This paper explores what it means to understand, how we evaluate it, and how that extends to AI. I argue that the methods used to evaluate understanding in humans test the same capacity as generalization metrics in machine learning: reliably solving problems not seen before. This capacity is achieved by manipulating inputs according to rules shared across an input space. The formation of these shared rules is described as compression, which is conceptually useful for identifying the presence of …
Read moreThis paper explores what it means to understand, how we evaluate it, and how that extends to AI. I argue that the methods used to evaluate understanding in humans test the same capacity as generalization metrics in machine learning: reliably solving problems not seen before. This capacity is achieved by manipulating inputs according to rules shared across an input space. The formation of these shared rules is described as compression, which is conceptually useful for identifying the presence of understanding in a system. Understanding as compression presents a tangible explanation for the generalization abilities of AI models, and the account is mechanistically tractable. Grokking in machine learning exposes the transition from memorization to generalization, and intrinsic dimensionality tracks the compression that drives it. For a society deciding whether to trust AI with critical tasks, this reframes machine understanding as something measurable rather than a metaphysical property to be debated.
