The Good Regulator Theorem and the Internal Model Principle are sometimes cited as mathematical proofs that an agent needs an internal model of the world in order to have an optimal policy. However, these principles rely on a definition of “internal model" that is far too permissive, applying even to cases of systems that do not use an internal model. As a result, these principles do not provide evidence (let alone a proof) that internal models are necessary. The paper also diagnoses what is mis…
Read moreThe Good Regulator Theorem and the Internal Model Principle are sometimes cited as mathematical proofs that an agent needs an internal model of the world in order to have an optimal policy. However, these principles rely on a definition of “internal model" that is far too permissive, applying even to cases of systems that do not use an internal model. As a result, these principles do not provide evidence (let alone a proof) that internal models are necessary. The paper also diagnoses what is missing in the GRT and IMP definitions of internal model, which is that models need to make predictions that represent variables in the target system (and these representations need to be usable by an agent so as to guide behavior).
