Herman Cappelen

This paper presents an argument that certain AI safety measures, rather than mitigating existential risk, may instead exacerbate it. Under certain key assumptions - the inevitability of AI failure, the expected correlation between an AI system's power at the point of failure and the severity of the resulting harm, and the tendency of safety measures to enable AI systems to become more powerful before failing - safety efforts have negative expected utility. The paper examines three response strategies: Optimism, Mitigation, and Holism. Each faces challenges stemming from intrinsic features of the AI safety landscape that we term Bottlenecking, the Perfection Barrier, and Equilibrium Fluctuation. The surprising robustness of the argument forces a re-examination of core assumptions around AI safety and points to several avenues for further research.

This paper assembles examples and considerations bearing on such questions as the following. Are statements to the effect that someone is too young or that someone is old enough always to be understood in terms of someone's being too young or too old for such-and-such-for example, for them to join a particular organization? And when a 'such-and-such' has been specified, is it always at least tacitly modal in force-in the case just given, too young or old enough to be able to join the organization? These questions are explored by means of a critical examination of the answers given to them by Eric Nelson in a 1980 paper on the subject, with part of the intention being to rescue Nelson's thoughtful discussion from the oblivion into which it appears to have fallen, judging by more recent contributions on the subject by semanticists.