-
9Reward tampering problems and solutions in reinforcement learning: a causal influence diagram perspectiveSynthese 198 (Suppl 27): 6435-6467. 2021.Can humans get arbitrarily capable reinforcement learning agents to do their bidding? Or will sufficiently capable RL agents always find ways to bypass their intended objectives by shortcutting their reward signal? This question impacts how far RL can be scaled, and whether alternative paradigms must be developed in order to build safe artificial general intelligence. In this paper, we study when an RL agent has an instrumental goal to tamper with its reward process, and describe design principl…Read more
-
216Universal Agent Mixtures and the Geometry of IntelligenceAistats. 2023.Inspired by recent progress in multi-agent Reinforcement Learning (RL), in this work we examine the collective intelligent behaviour of theoretical universal agents by introducing a weighted mixture operation. Given a weighted set of agents, their weighted mixture is a new agent whose expected total reward in any environment is the corresponding weighted average of the original agents' expected total rewards in that environment. Thus, if RL agent intelligence is quantified in terms of performanc…Read more
-
4Classification by decomposition: a novel approach to classification of symmetric $$2\times 2$$ games (review)Theory and Decision 93 (3): 463-508. 2022.In this paper, we provide a detailed review of previous classifications of $$2\times 2$$ 2 × 2 games and suggest a mathematically simple way to classify the symmetric $$2\times 2$$ 2 × 2 games based on a decomposition of the payoff matrix into a cooperative and a zero-sum part. We argue that differences in the interaction between the parts is what makes games interesting in different ways. Our claim is supported by evolutionary computer experiments and findings in previous literature. In additio…Read more
-
238Reward-Punishment Symmetric Universal IntelligenceIn AGI, . 2021.Can an agent's intelligence level be negative? We extend the Legg-Hutter agent-environment framework to include punishments and argue for an affirmative answer to that question. We show that if the background encodings and Universal Turing Machine (UTM) admit certain Kolmogorov complexity symmetries, then the resulting Legg-Hutter intelligence measure is symmetric about the origin. In particular, this implies reward-ignoring agents have Legg-Hutter intelligence 0 according to such UTMs.
-
5Universal Algorithmic Intelligence: A Mathematical Top-Down ApproachIn Ben Goertzel & Cassio Pennachin (eds.), Artificial General Intelligence, Springer Verlag. pp. 227-290. 2007.Sequential decision theory formally solves the problem of rational agents in uncertain worlds if the true environmental prior probability distribution is known. Solomonoff's theory of universal induction formally solves the problem of sequence prediction for unknown prior distribution. We combine both ideas and get a parameter-free theory of universal Artificial Intelligence. We give strong arguments that the resulting AIXI model is the most intelligent unbiased agent possible. We outline how th…Read more
-
7Proceedings of the Second Conference on Artificial General Intelligence (edited book)Atlantis Press. 2009.The Conference on Artificial General Intelligence is the only major conference series devoted wholly and specifically to the creation of AI systems possessing general intelligence at the human level and ultimately beyond. Its second installation, AGI-09, in Arlington, Virginia, March 6-9, 2009, attracted 67 paper submissions, which is a substantial increase from the previous year. Of these submissions, 33 (i.e., 49%) were accepted as full papers for presentation at the conference. Additional 13 …Read more
-
115A Philosophical Treatise of Universal InductionEntropy 13 (6): 1076-1136. 2011.Understanding inductive reasoning is a problem that has engaged mankind for thousands of years. This problem is relevant to a wide range of fields and is integral to the philosophy of science. It has been tackled by many great minds ranging from philosophers to scientists to mathematicians, and more recently computer scientists. In this article we argue the case for Solomonoff Induction, a formal inductive framework which combines algorithmic information theory with the Bayesian framewor…Read more
-
643One decade of universal artificial intelligenceIn Pei Wang & Ben Goertzel (eds.), Theoretical Foundations of Artificial General Intelligence, Springer. pp. 67--88. 2012.The first decade of this century has seen the nascency of the first mathematical theory of general artificial intelligence. This theory of Universal Artificial Intelligence (UAI) has made significant contributions to many theoretical, philosophical, and practical AI questions. In a series of papers culminating in book (Hutter, 2005), an exciting sound and complete mathematical model for a super intelligent agent (AIXI) has been developed and rigorously analyzed. While nowadays most AI researcher…Read more
-
267A Complete Theory of Everything (will be subjective)Algorithms 3 (4): 329-350. 2010.Increasingly encompassing models have been suggested for our world. Theories range from generally accepted to increasingly speculative to apparently bogus. The progression of theories from ego- to geo- to helio-centric models to universe and multiverse theories and beyond was accompanied by a dramatic increase in the sizes of the postulated worlds, with humans being expelled from their center to ever more remote and random locations. Rather than leading to a true theory of everything, thi…Read more
-
240Probabilities on Sentences in an Expressive LogicJournal of Applied Logic 11 (4): 386-420. 2013.Automated reasoning about uncertain knowledge has many applications. One difficulty when developing such systems is the lack of a completely satisfactory integration of logic and probability. We address this problem directly. Expressive languages like higher-order logic are ideally suited for representing and reasoning about structured knowledge. Uncertain knowledge can be modeled by using graded probabilities rather than binary truth-values. The main technical problem studied in this pape…Read more
-
157Can Intelligence Explode?Journal of Consciousness Studies 19 (1-2): 143-166. 2012.The technological singularity refers to a hypothetical scenario in which technological advances virtually explode. The most popular scenario is the creation of super-intelligent algorithms that recursively create ever higher intelligences. It took many decades for these ideas to spread from science fiction to popular science magazines and finally to attract the attention of serious philosophers. David Chalmers' (JCS 2010) article is the first comprehensive philosophical analysis of the singulari…Read more
-
401Universal intelligence: A definition of machine intelligenceMinds and Machines 17 (4): 391-444. 2007.A fundamental problem in artificial intelligence is that nobody really knows what intelligence is. The problem is especially acute when we need to consider artificial systems which are significantly different to humans. In this paper we approach this problem in the following way: we take a number of well known informal definitions of human intelligence that have been given by experts, and extract their essential features. These are then mathematically formalised to produce a general measure of i…Read more
-
283Observer Localization in Multiverse TheoriesIn Harald Fritzsch & K. K. Phua (eds.), Proceedings of the Conference in Honour of Murray Gell-Mann's 80th Birthday, World Scientific. 2010.The progression of theories suggested for our world, from ego- to geo- to helio-centric models to universe and multiverse theories and beyond, shows one tendency: The size of the described worlds increases, with humans being expelled from their center to ever more remote and random locations. If pushed too far, a potential theory of everything (TOE) is actually more a theories of nothing (TON). Indeed such theories have already been developed. I show that including observer localization i…Read more
-
Australian National UniversityResearcher
Acton, Australian Capital Territory, Australia
Areas of Interest
Philosophy of Probability |
General Philosophy of Science |