Sami Petersen

Robert Aumann influentially claimed that the completeness axiom in utility theory is neither descriptively accurate nor normatively compelling. Separately, it’s commonly accepted that dynamic consistency in individual decision-making is a demand of rationality. These assertions may conflict, however, even in mundane decision problems lacking any form of preference change. I characterise the most permissive preference-based choice rule which allows an agent with incomplete but transitive preferences to satisfy a strong form of dynamic consistency. The resulting behaviour is not generally rationalisable by a completed preference relation. I thereby present a challenge to—and conditions for—the simultaneous satisfaction of strong dynamic consistency and preservation of preferential gaps.

A principal wants to deploy an artificial intelligence (AI) system to perform some task. But the AI may be misaligned and pursue a conflicting objective. The principal cannot restrict its options or deliver punishments. Instead, the principal can (i) simulate the task in a testing environment and (ii) impose imperfect recall on the AI, obscuring whether the task being performed is real or part of a test. By committing to a testing mechanism, the principal can screen the misaligned AI during testing and discipline its behaviour in deployment. Increasing the number of tests allows the principal to screen or discipline arbitrarily well. The screening effect is preserved even if the principal cannot commit or if the agent observes information partially revealing the nature of the task. Without commitment, imperfect recall is necessary for testing to be helpful.