Alexey Turchin

Abstract. In this article is presented a model of the change of the probability of the global catastrophic risks in the world with exponentially evolving technologies. Increasingly cheaper technologies become accessible to a larger number of agents. Also, the technologies become more capable to cause a global catastrophe. Examples of such dangerous technologies are artificial viruses constructed by the means of synthetic biology, non-aligned AI and, to less extent, nanotech and nuclear proliferation. The model shows at least double exponential growth of the probability of the global catastrophe which means that the accumulated probability of the catastrophe will grow from negligible to overwhelming at the period of a few doubling times of the technological capabilities. For biotech and AI, such doubling time roughly corresponds to the doubling period of Moore’s law and its analogues in other technologies and is around 2 years. Thus the global catastrophe in the exponential technologies world will most likely happen during a “dangerous decade”. Such a dangerous decade could start as early as in the 2020s. We also found that the double exponential growth in the model makes the model less sensitive to its initial conditions, like the number of dangerous agents or initial probabilities constants. The model also shows that smaller catastrophes could happen earlier than larger ones, and such a smaller catastrophe may be able to stop the technological growth before larger ones become possible, thus global risks will be self-limiting. However, if the growth of the number of dangerous agents will be very quick, the multiple smaller catastrophes could happen simultaneously and will be equal to a global catastrophe.

Does a single mechanism of aging exit? Most scientists have their own pet theories about what is aging, but the lack of generally accepted theory is mind-blowing. Here we suggest an explanation: evolution works against unitary mechanism of aging because it equalizes ‘warranty period’ of different resilience systems. Therefore, we need life-extension methods that go beyond fighting specific aging mechanisms: such as using a combination of geroprotectors or repair-fixing bionanorobots controlled by AI.

Humanity may underestimate the rate of natural global catastrophes because of the survival bias (“anthropic shadow”). But the resulting reduction of the Earth’s future habitability duration is not very large in most plausible cases (1-2 orders of magnitude) and thus it looks like we still have at least millions of years. However, anthropic shadow implies anthropic fragility: we are more likely to live in a world where a sterilizing catastrophe is long overdue and could be triggered by unexpectedly small human actions. In the same way, an over-inflated toy balloon, which will soon burst, is very fragile. Anthropic fragility can manifest itself in the higher chances of runaway global warming. It has often been suggested that the Earth's atmosphere remained life-supporting for billions of years by sheer chance. Therefore, the survival bias can be strong. It is also known that Earth-like water worlds could experience transitions into deadly moisture greenhouse (mean T = 65C). This means that relatively small anthropogenic actions could put the climate above an unpredictable tipping point, which could lead to the moisture greenhouse. Thus, it is necessary to carry out urgent geoengineering studies and prepare to prevent an unexpected climate catastrophe. There are three main counterarguments against the existence of the anthropic shadow: self-indication assumption (SIA), past observers and the Gaia hypothesis; we show that they fail. It was proposed that SIA exactly compensates the anthropic shadow as an observer unlikely to find herself in a world with a strong anthropic shadow; however, there is a baseline level of the anthropic shadow for all habitable planets, similar to the rate of evolutionary transitions like abiogenesis. There are no “past observers” as qualified observers appeared only 50 years ago. Gaia hypothesis assumes existence of self-stabilizing feedback in climate, but new types of events like quick CO2 growth could override its coping ability. We present a list of other catastrophes that may have been underestimated because of the anthropic shadow, including collider catastrophes, nuclear war and even an alien invasion. We also hypothesized that human intelligence is more likely to emerge in an unstable world that is nearing its end and thus we get a new form of Doomsday argument.

In this article, I present a view on the future of nuclear war which takes into account the expected technological progress as well as global political changes. There are three main directions in which technological progress in nuclear weapons may happen: a) Many gigaton scale weapons. b) Cheaper nuclear bombs which are based on the use of the reactor-grade plutonium, laser isotope separation or are hypothetical pure fusion weapons. Also, advanced nanotechnology will provide the ability to quickly build large nuclear arsenals and AI could be used in designing, manufacturing and nuclear strategy planning. c) Specialized nukes like nuclear-powered space lasers, hafnium bombs and nuclear-powered space ships as kinetic weapons. Meanwhile, the nuclear war strategy also has changed as the bipolar world has ended and as new types of weapons are becoming available. The first strategy change is that Doomsday weapons for blackmail will become attractive for weaker countries which can’t use ICBM to penetrate the anti-missile defense of the enemies. Secondly, the cheaper nukes will become available to smaller actors, who may be involved in “worldwide guerilla”. Cheaper nukes and a larger number of actors also encourage nuclear terrorism and anonymous or false-flag warfare. This will result in the normalization of the use of nuclear weapons in war and growing nuclear arsenals, now distributed between many countries. The third change of the strategy is the use of nukes not against primary military targets but on other objects, which could increase their effects: nuclear power plants, supervolcanos, EMP, tsunamis, taiga fires for global cooling, and even more hypothetical things like asteroid deflection to Earth. All these unconventional means and strategies of nuclear war could become available somewhere in the 21st century and may cause or contribute to the civilizational collapse and even human extinction.

In AI safety research, the median timing of AGI arrival is often taken as a reference point, which various polls predict to happen in the middle of 21 century, but for maximum safety, we should determine the earliest possible time of Dangerous AI arrival. Such Dangerous AI could be either AGI, capable of acting completely independently in the real world and of winning in most real-world conflicts with humans, or an AI helping humans to build weapons of mass destruction, or a national state coupled with AI-based government system. In this article, I demonstrate that the earliest timing of Dangerous AI, corresponding to 10 per cent of its arrival probability, is before 2030. Several partly independent sources of information are in accordance: 1. The growth of the hardware available for AI research makes human-brain-equivalents of compute available for AI research in the 2020s. It is fuelled by specialized AI-chips, the use of many chips in one processing unit, and the larger research budgets, among other things. 2. The neural network performance and other characteristics, like the number of parameters, is quickly increasing every year, and extrapolating this tendency suggests that roughly human-level performance in a few years, around 2025. 3. Expert polls show around 10 per cent of the probability of an early appearance of artificial general intelligence (AGI) in the next decade, that is, before 2030. 4. Hyperbolic growth in different big history models converges around 2025-2030 (the technological singularity). 5. Anthropic arguments (similar to the Doomsday argument) suggest that qualified observers are more likely to appear near the end of the AI research epoch, as the number of such observers grew exponentially. This number doubles every 5-10 years, and thus we are likely to find ourselves around a decade before the end of AI research, which will happen consequently around 2030.

With the fast pace of AI development, the problem of preventing its global catastrophic risks arises. However, no satisfactory solution has been found. From several possibilities, the confinement of AI in a box is considered as a low-quality possible solution for AI safety. However, some treacherous AIs can be stopped by effective confinement if it is used as an additional measure. Here, we proposed an idealized model of the best possible confinement by aggregating all known ideas in the field of AI boxing. We model the confinement based on the principles used in the safety engineering of nuclear power plants. We show that AI confinement should be implemented in several levels of defense. These levels include 1) AI design in fail-safe manner 2) limiting its capabilities, preventing self-improving and circuit breakers on treacherous turn 3) isolation from the outside world and, lastly, as the final hope 4) outside measures oriented on stopping AI in the wild. We demonstrate that the substantial number (more than 50 ideas listed in the article) of mutually independent measures could provide a relatively high probability of the containment of a human-level AI but may be not sufficient to preserve runaway of superintelligent AI. Thus, these measures will work only if they are used to prevent superintelligent AI creation, but not for containing superintelligence. We suggest that there could be a safe operation threshold, on which AI is useful, but is not able to hack containment system from the inside, the same way as a safe level of chain reaction inside nuclear power plants is maintained. However, overall, a failure of the confinement is inevitable, so we need to use the full AGI limited number of times (AI-ticks).

Lucid dreaming (LD) is a fun and interesting activity, but most participants have difficulties in attaining lucidity, retaining it during the dream, concentrating on the needed task and remembering the results. This motivates to search for a new way to enhance lucid dreaming via different induction techniques, including chemicals and electric brain stimulation. However, results are still unstable. An alternative approach is to reach the lucid dreaming-like states via altered state of consciousness not related to dreaming. Several methods such as guided visualization, internal dialog, creative writing, hypnosis, hypnagogia, daydreaming, DMT trips, voice dialog, shamanic journey, rebirthing, and “forcing” tulpas can help in attaining such states. One of the most promising of them is Jungian “active imagination” (AIM) technique, which allows unconscious content to build up inside some mental frames. This article explores the hypothesis of replacing lucid dreaming research with active imagination, and the conditions and ways to accomplish it. Method: An open label pilot experiment was performed in 2004-2005 in Moscow, Russia with 100 participants. Results: The results show that there are two groups of people: ones with “visual imagination screen” and others have “mental imagination screen”. AIM works perfectly as a replacement for lucid dreams only for the first group of people. For the second group, it created interesting content, but not visual or emotional intensity equal to enter lucid dreaming like state. No known instruments helped to move the person from one group to another. The first group consisted of young females, while the second mostly contained males with rational and mathematical type of personality. Conclusion: AIM partly works as a replacement for LD, as it works great only for half of people, and it requires a sitter. However, AIM outperforms LD in reliability and availability in any circumstance: it could be performed even by text chat or in a crowd. It is also better than LD in retaining concentration on topic and the easiness of memorizing the results (which could be recorded). Self-performed AIM is less effective. AIM can be improved by intelligent chat bots as sitters and weak brain stimulation that can increase the probability of attaining something like hypnogogic state.

This article explores theoretical conditions necessary for “quantum immortality” as well as its possible practical implications. It is demonstrated that QI is a particular case of “multiverse immortality”, which is based on two main assumptions: the very large size of the universe ; and a copy-friendly theory of personal identity. It is shown that a popular objection about lowering of the world-share of an observer in the case of QI does not succeed, as the world-share decline could be compensated by merging timelines for the simpler minds, and because some types of personal preferences are not dependent on such changes. Despite large uncertainty about the truth of MI, it has appreciable practical consequences for some important outcomes like suicide and aging. The article demonstrates that MI could be used to significantly increase the expected subjective probability of success of risky life extension technologies, such as cryonics, but that it makes euthanasia impractical because of the risk of eternal suffering. Euthanasia should be replaced with cryothanasia, i.e. cryopreservation after voluntary death. Another possible application of MI is as a last chance to survive a global catastrophe. MI could be considered a Plan D for reaching immortality, where Plan A consists of survival until the development of beneficial Artificial Intelligence capable of fighting aging, Plan B employs cryonics, and Plan C is digital immortality.

The long unbearable sufferings in the past and agonies experienced in some future timelines in which a malevolent AI could torture people for some idiosyncratic reasons (s-risks) is a significant moral problem. Such events either already happened or will happen in causally disconnected regions of the multiverse and thus it seems unlikely that we can do anything about it. However, at least one pure theoretic way to cure past sufferings exists. If we assume that there is no stable substrate of personal identity and thus a copy equals original, then by creating many copies of the next observer-moment of a person in pain in which he stops suffer, we could create indexical uncertainty in her future location and thus effectively steal her consciousness from her initial location and immediately relieve her sufferings. However, to accomplish this for people who have already died, we need to perform this operation for all possible people thus requiring enormous amounts of computations. Such computation could be performed by the future benevolent AI of Galactic scale. Many such AIs could cooperate acausally by distributing parts of the work between them via quantum randomness. To ensure their success, they need to outnumber all possible evil AIs by orders of magnitude, and thus they need to convert most of the available matter into computronium in all universes where they exist and cooperate acausally across the whole multiverse. Another option for curing past suffering is the use of wormhole time-travel to send a nanobot in the past which will, after a period of secret replication, collect the data about people and secretly upload them when their suffering becomes unbearable.

Abstract. The presumptuous philosopher (PP) thought experiment lends more credence to the hypothesis which postulates the existence of a larger number of observers than other hypothesis. The PP was suggested as a purely speculative endeavor. However, there is a class of real world observer-selection effects where it could be applied, and one of them is the possibility of interstellar panspermia (IP). There are two types of anthropic reasoning: SIA and SSA. SIA implies that my existence is an argument that larger total number of observers exists; SSA implies that I should find myself in a region with larger number of observers. However, as S. Armstrong showed, SIA can’t distinguish between different ways how larger number of observers appeared, so it can’t favor IP compared with other ways to get many observers. SSA application here is less controversial as it tells only about relation between regions size: e.g. I am more likely to live in a larger country than in a small one, conditioning that the total number of small countries is also small. Anthropic considerations suggest that the universes in the multiverse with interstellar panspermia will have orders of magnitude more civilizations than universes without IP, and thus we are likely to be in such a universe. This is a strong counterargument against a variant of the Rare Earth hypothesis based on the difficulty of abiogenesis: even if abiogenesis is difficult, IP will disseminate life over billions of planets, meaning we are likely to find ourselves in a universe where IP has happened. This implies that there should be many planets with life in our galaxy, and renders the Fermi paradox even sharper. Either the Great Filter is ahead of us and there are high risks of anthropogenic extinction, or there are many alien civilizations nearby and of the same age as ours—which is itself a global catastrophic risk, as in that case the wave of alien colonization could arrive between 25-500 ky from now.

Abstract: After 2017 NY Times publication, the stigma of the scientific discussion of the problem of so-called UAP (Unidentified Aerial Phenomena) was lifted. Now the question arises: how UAP will affect the future of humanity, and especially, the probability of the global catastrophic risks? To answer this question, we assume that the Nimitz case in 2004 was real and we will suggest a classification of the possible explanations of the phenomena. The first level consists of mundane explanations: hardware glitches, malfunction, hoaxes and lies. The second level involves explanations which are not changing completely our model of the world, but only updating it: new military tech, large psyop operation or new physical effect like ball lightning. The third level of explanations requires a complete overhaul of our world model and includes a large set of fantastic hypothesis: alien starships, interdimensional beings, glitches in the matrix, projections from the collective unconsciousness, Boltzmann brain’s experiences etc. The higher is the level of the hypothesis, the less probable it is, but the bigger is its consequences for possible global catastrophic risks. Thus, “integrating” over the field of all possible explanations, we find that UAP increases catastrophic risks. Technological progress increases our chances of direct confrontation with the phenomena. If the phenomenon has intelligence, the change of its behavior could be unexpected after we pass some unknown threshold. But we could lower the risks if we assume some reasonable policy of escaping confrontations.

The problem of surviving the end of the observable universe may seem very remote, but there are several reasons it may be important now: a) we may need to define soon the final goals of runaway space colonization and of superintelligent AI, b) the possibility of the solution will prove the plausibility of indefinite life extension, and с) the understanding of risks of the universe’s end will help us to escape dangers like artificial false vacuum decay. A possible solution depends on the type of the universe’s ending that may be expected: very slow heat death or some abrupt end, like a Big Rip or Big Crunch. We have reviewed the literature and identified several possible ways of survival the end of the universe, and also suggest several new ones. There are seven main approaches to escape the end of the universe: use the energy of the catastrophic process for computations, move to a parallel world, prevent the end, survive the end, manipulate time, avoid the problem entirely or find some meta-level solution.

Abstract: The field of life extension is full of ideas but they are unstructured. Here we suggest a comprehensive strategy for reaching personal immortality based on the idea of multilevel defense, where the next life-preserving plan is implemented if the previous one fails, but all plans need to be prepared simultaneously in advance. The first plan, plan A, is the surviving until advanced AI creation via fighting aging and other causes of death and extending one’s life. Plan B is cryonics, which starts if plan A fails, and assumes cryopreservation of the brain until technical capabilities to return it to life appear. Plan C is digital immortality in the sense of collecting data about the person now so future AI will be able to recreate a model of a person. Plan D is the hope based on some unlikely scenarios of infinite survival, like so-called “quantum immortality”. All these plans have personal and social perspective. The personal aspect means efforts of the increasing chances of personal survival via taking care about one’s own health, signing cryocontract or collecting digital immortality data. The social aspect means the participation in collective work towards creation and increase of the availability of life extension technologies, which includes funding scientific research, promotion of life extension value and direct performing of research and implementation, as well as preventing global catastrophic risk. All plans converge at the end, as their result is the indefinite survival as an uploaded mind inside an ecosystem, created by a superintelligent AI.

Purpose This paper aims to formalize long-term trajectories of human civilization as a scientific and ethical field of study. The long-term trajectory of human civilization can be defined as the path that human civilization takes during the entire future time period in which human civilization could continue to exist. Design/methodology/approach This paper focuses on four types of trajectories: status quo trajectories, in which human civilization persists in a state broadly similar to its current state into the distant future; catastrophe trajectories, in which one or more events cause significant harm to human civilization; technological transformation trajectories, in which radical technological breakthroughs put human civilization on a fundamentally different course; and astronomical trajectories, in which human civilization expands beyond its home planet and into the accessible portions of the cosmos. Findings Status quo trajectories appear unlikely to persist into the distant future, especially in light of long-term astronomical processes. Several catastrophe, technological transformation and astronomical trajectories appear possible. Originality/value Some current actions may be able to affect the long-term trajectory. Whether these actions should be pursued depends on a mix of empirical and ethical factors. For some ethical frameworks, these actions may be especially important to pursue.

One of the possible solutions of the Fermi paradox is that all civilizations go extinct because they hit some Late Great Filter. Such a universal Late Great Filter must be an unpredictable event that all civilizations unexpectedly encounter, even if they try to escape extinction. This is similar to the “Death in Damascus” paradox from decision theory. However, this unpredictable Late Great Filter could be escaped by choosing a random strategy for humanity’s future development. However, if all civilizations act randomly, this could be the actual cause of universal extinction, as no one will choose the most optimal strategy. Using a meta-random strategy, where the decision to start random actions is also made randomly, solves this.

Abstract. Boltzmann brains (BBs) are minds which randomly appear as a result of thermodynamic or quantum fluctuations. In this article, the question of if we are BBs, and the observational consequences if so, is explored. To address this problem, a typology of BBs is created, and the evidence is compared with the Simulation Argument. Based on this comparison, we conclude that while the existence of a “normal” BB is either unlikely or irrelevant, BBs with some ordering may have observable consequences. There are two types of such ordered BBs: Boltzmannian typewriters (including Boltzmannian simulations), and chains of observer moments. Notably, the existence or non-existence of BBs may have practical applications for measuring the size of the universe, achieving immortality, or even manipulating the observed probability of events. Disclaimer and trigger warning: some people have emotional breakdowns when thinking about topics described in the article, especially the “flux universe”. However, everything eventually adds up to normality.

Abstract: In the last decade, an urban legend about “glitches in the matrix” has become popular. As it is typical for urban legends, there is no evidence for most such stories, and the phenomenon could be explained as resulting from hoaxes, creepypasta, coincidence, and different forms of cognitive bias. In addition, the folk understanding of probability does not bear much resemblance to actual probability distributions, resulting in the illusion of improbable events, like the “birthday paradox”. Moreover, many such stories, even if they were true, could not be considered evidence of glitches in a linear-time computer simulation, as the reported “glitches” often assume non-linearity of time and space—like premonitions or changes to the past. Different types of simulations assume different types of glitches; for example, dreams are often very glitchy. Here, we explore the theoretical conditions necessary for such glitches to occur and then create a typology of so-called “GITM” reports. One interesting hypothetical subtype is “viruses in the matrix”, that is, self-replicating units which consume computational resources in a manner similar to transposons in the genome, biological and computer viruses, and memes.

The goal of the article is to explore what is the most probable type of simulation in which humanity lives (if any) and how this affects simulation termination risks. We firstly explore the question of what kind of simulation in which humanity is most likely located based on pure theoretical reasoning. We suggest a new patch to the classical simulation argument, showing that we are likely simulated not by our own descendants, but by alien civilizations. Based on this, we provide classification of different possible simulations and we find that simpler, less expensive and one-person-centered simulations, resurrectional simulations, or simulations of the first artificial general intelligence’s (AGI’s) origin (singularity simulations) should dominate. Also, simulations which simulate the 21st century and global catastrophic risks are probable. We then explore whether the simulation could collapse or be terminated. Most simulations must be terminated after they model the singularity or after they model a global catastrophe before the singularity. Undeniably observed glitches, but not philosophical speculations could result in simulation termination. The simulation could collapse if it is overwhelmed by glitches. The Doomsday Argument in simulations implies termination soon. We conclude that all types of the most probable simulations except resurrectional simulations are prone to termination risks in a relatively short time frame of hundreds of years or less from now.

Abstract: The field of artificial general intelligence (AGI) safety is quickly growing. However, the nature of human values, with which future AGI should be aligned, is underdefined. Different AGI safety researchers have suggested different theories about the nature of human values, but there are contradictions. This article presents an overview of what AGI safety researchers have written about the nature of human values, up to the beginning of 2019. 21 authors were overviewed, and some of them have several theories. A theory classification method is suggested, where the theories are judged according to the level of their complexity and behaviorists-internalists scale, as well as the level of their generality-humanity. We suggest that a multiplicity of well-supported theories means that the nature of human values is difficult to define, and some meta-level theory is needed.

Abstract. The main current approach to the AI safety is AI alignment, that is, the creation of AI whose preferences are aligned with “human values.” Many AI safety researchers agree that the idea of “human values” as a constant, ordered sets of preferences is at least incomplete. However, the idea that “humans have values” underlies a lot of thinking in the field; it appears again and again, sometimes popping up as an uncritically accepted truth. Thus, it deserves a thorough deconstruction, which I will do by listing and analyzing comprehensively the hidden assumptions of the idea that “humans have values.” This deconstruction of human values will be centered around the following ideas: “Human values” are useful descriptions, but not real objects; “human values” are bad predictors of behavior; the idea of a “human value system” has flaws; “human values” are not good by default; and human values cannot be separated from human minds. The method of analysis is listing hidden assumptions on which the idea of “human values” is built. I recommend that either the idea of “human values” should be replaced with something better for the goal of AI safety, or at least be used very cautiously. The approaches to AI safety which don’t use the idea of human values at all may require more attention, like the use of full brain models, boxing, and capability limiting.

Abstract: In the future, it will be possible to create advance simulations of ancestor in computers. Superintelligent AI could make these simulations very similar to the real past by creating a simulation of all of humanity. Such a simulation would use all available data about the past, including internet archives, DNA samples, advanced nanotech-based archeology, human memories, as well as text, photos and videos. This means that currently living people will be recreated in such a simulation, and in some sense, “resurrected”. Such “resurrectional simulation” could be deliberately created just for this goal: to return to life all people who have ever lived. The main technical problem of such simulation will be uncertainty about the past, which increases exponentially for more remote times. Such problem could be partly addressed by “acausal trade” between different branches of the multiverse, which will create slightly different versions of the simulation using a quantum randomness generator. Such trade will result in resurrection of all possible people (including those who existed in other branches). Ethical problems of such a resurrectional simulation include: a) possible resurrection of some people against their will; b) such simulation may create additional suffering; с) such simulation could be used by hostile AI to return people to life and then torture them. In this work, I explore preliminary ideas about how to address these problems.

Abstract: As there are no visible ways to create safe self-improving superintelligence, but it is looming, we probably need temporary ways to prevent its creation. The only way to prevent it, is to create special AI, which is able to control and monitor all places in the world. The idea has been suggested by Goertzel in form of AI Nanny, but his Nanny is still superintelligent and not easy to control, as was shown by Bensinger at al. We explore here the ways to create the safest and simplest form of AI, which may work as AI Nanny. Such AI system will be enough to solve most problems, which we expect the AI will solve, including control of robotics, acceleration of the medical research, but will present less risk, as it will be less different from humans. As AI police, it will work as operation system for most computers, producing world surveillance system, which will be able to envision and stop any potential terrorists and bad actors in advance. As uploading technology is lagging, and neuromorphic AI is intrinsically dangerous, the most plausible way to human-based AI Nanny is either functional model of the human mind or a Narrow-AI empowered group of people.

Many global catastrophic and existential risks (X-risks) threaten the existence of humankind. There are also many ideas for their prevention, but the meta-problem is that these ideas are not structured. This lack of structure means it is not easy to choose the right plan(s) or to implement them in the correct order. I suggest using a “Plan A, Plan B” model, which has shown its effectiveness in planning actions in unpredictable environments. In this approach, Plan B is a backup option, implemented if Plan A fails. In the case of global risks, Plan A is intended to prevent a catastrophe and Plan B to survive it, if it is not avoided. Each plan has similar stages: analysis, planning, funding, low-level realization and high-level realization. Two variables—plans and stages—provide an effective basis for classification of all possible X-risks prevention methods in the form of a two-dimensional map, allowing the choice of optimal paths for human survival. I have created a framework for estimating the utility of various prevention methods based on their probability of success, the chances that they will be realized in time, their opportunity cost, and their risk. I also distinguish between top-down and bottom-up approaches.

Abstract: Global chemical contamination is an underexplored source of global catastrophic risks that is estimated to have low a priori probability. However, events such as pollinating insects’ population decline and lowering of the human male sperm count hint at some toxic exposure accumulation and thus could be a global catastrophic risk event if not prevented by future medical advances. We identified several potentially dangerous sources of the global chemical contamination, which may happen now or could happen in the future: autocatalytic reactions, exposure to multiple subthreshold sources, and long-term unintended consequences, arising from both natural and bioengineered sources. We list several especially dangerous chemicals—dioxin, organiс compounds, and toxic heavy metals. We also discuss the features of such dangerous chemicals—molecules that can stay in the biosphere for a long time and affect it over time. We explore several social processes and scenarios where global chemical contamination becomes possible: large natural catastrophe like meteorite impact, supervolcano eruption, new ways of predicting properties of the chemicals via machine learning and their manufacturing via synthetic biology, uncontrolled “capitalistic” economic development with a corresponding large waste production, quick adoption of many chemicals with unknown long-term properties and unintended side-effects. These are all low probability, so work on other global catastrophic risks should be prioritized, but chemical risks could exacerbate other types of catastrophe contributing to social collapse

Abstract: Four main forms of Doomsday Argument (DA) exist—Gott’s DA, Carter’s DA, Grace’s DA and Universal DA. All four forms use different probabilistic logic to predict that the end of the human civilization will happen unexpectedly soon based on our early location in human history. There are hundreds of publications about the validity of the Doomsday argument. Most of the attempts to disprove the Doomsday Argument have some weak points. As a result, we are uncertain about the validity of DA proofs and rebuttals. In this article, a meta-DA is introduced, which uses the idea of logical uncertainty over the DA’s validity estimated based on a virtual prediction market of the opinions of different scientists. The result is around 0.4 for the validity of some form of DA, and even smaller for “Strong DA”, which predicts the end of the world in the near term. We discuss many examples of the validity of the DA in real life as an instrument to prove it “experimentally”. We also show that DA becomes strongest if it is based on the idea of the “natural reference class” of observers, that is, the observers who know about the DA (i.e. a Self-Referenced DA). Such a DA predicts that there is a high probability of a global catastrophe with human extinction in the 21st century, which aligns with what we already know based on analysis of different technological risks.

In this article, a classification of the global catastrophic risks connected with the possible existence (or non-existence) of extraterrestrial intelligence is presented. If there are no extra-terrestrial intelligences (ETIs) in our light cone, it either means that the Great Filter is behind us, and thus some kind of periodic sterilizing natural catastrophe, like a gamma-ray burst, should be given a higher probability estimate, or that the Great Filter is ahead of us, and thus a future global catastrophe is high probability. If ETIs exist in our light cone, it is most likely they exist in the form of ET artificial general intelligence (ETAI). ETAI may be traveling at near-light speed as an explosive wave of intelligence in the form of some nano-mechanisms which could reach us at any moment; may send us potentially dangerous messages with AI-virus payloads; or be near or on Earth in some dormant form. This dormant form could be nanobots, which could act as berserkers, and be triggered when humans unwittingly cross some threshold. If ETIs are nearby and less advanced, they could find our messaging extra-terrestrial intelligence (METI) messages and send their “fleet” to or direct weapons toward Earth. Even extinct ETIs could have left dangerous remnants in the form of AI systems that our civilization may encounter. If our existence is a simulation, this could be run by ETIs. Even a false belief in the existence of ETIs could potentially fuel millennial sects. If humanity is the first intelligence to emerge, it may kill or prevent the existence of all future potential ETI, a different type of catastrophe. Several options to prevent catastrophic risks are connected with ETI: sending requests for help, using random strategies to escape the Fermi paradox, Great Filter prediction, or the hope that ETIs will find the remains of our extinct civilization and resurrect it.

Abstract: Advances in new technologies create new ways to stimulate the pleasure center of the human brain via new chemicals, direct application of electricity, electromagnetic fields, “reward hacking” in games and social networks, and in the future, possibly via genetic manipulation, nanorobots and AI systems. This may have two consequences: a) human life may become more interesting, b) humans may stop participating in any external activities, including work, maintenance, reproduction, and even caring for their own health, which could slowly contribute to a decline in human populations. As wireheading is not self-limiting, it may push the addict to his/her limits and ultimately result in life-threatening behavior. Mass unemployment and the provision of basic income would likely feed the industry of pleasures. DIY synthetic biology and computers may create the possibility of homemade very addictive drugs through various routes, like insertion of a morphine-producing gene in plants or stimulation of the brain via electromagnetic fields, which may not be possible to control via limiting the supply chain. If “wireheading knowledge” becomes freely available, it could create a reward-hacking epidemic larger than the current U.S. opioid crisis and possibly contribute to a global civilizational decline.

Purpose Islands have long been discussed as refuges from global catastrophes; this paper will evaluate them systematically, discussing both the positives and negatives of islands as refuges. There are examples of isolated human communities surviving for thousands of years on places like Easter Island. Islands could provide protection against many low-level risks, notably including bio-risks. However, they are vulnerable to tsunamis, bird-transmitted diseases, and other risks. This article explores how to use the advantages of islands for survival during global catastrophes. Methodology: Preliminary horizon scanning based on the application of the research principles established in the previous global catastrophic literature to the existing geographic data. Findings The large number of islands on Earth, and their diverse conditions, increase the chance that one of them will provide protection from a catastrophe. Additionally, this protection could be increased if an island were used as a base for a nuclear submarine refuge combined with underground bunkers, and/or extremely long-term data storage. The requirements for survival on islands, their vulnerabilities, and ways to mitigate and adapt to risks are explored. Several existing islands, suitable for the survival of different types of risk, timing, and budgets, are examined. Islands suitable for different types of refuges and other island-like options that could also provide protection are also discussed. Originality/value The possible use of islands as refuges from social collapse and existential risks has not been previously examined systematically. This article contributes to the expanding research on survival scenarios.

Abstract: As there are no currently obvious ways to create safe self-improving superintelligence, but its emergence is looming, we probably need temporary ways to prevent its creation. The only way to prevent it is to create a special type of AI that is able to control and monitor the entire world. The idea has been suggested by Goertzel in the form of an AI Nanny, but his Nanny is still superintelligent, and is not easy to control. We explore here ways to create the safest and simplest form of AI which may work as an AI Nanny, that is, a global surveillance state powered by a Narrow AI, or AI Police. A similar but more limited system has already been implemented in China for the prevention of ordinary crime. AI police will be able to predict the actions of and stop potential terrorists and bad actors in advance. Implementation of such AI police will probably consist of two steps: first, a strategic decisive advantage via Narrow AI created by an intelligence services of a nuclear superpower, and then ubiquitous control over potentially dangerous agents which could create unauthorized artificial general intelligence which could evolve into Superintelligence.

A classification of the global catastrophic risks of AI is presented, along with a comprehensive list of previously identified risks. This classification allows the identification of several new risks. We show that at each level of AI’s intelligence power, separate types of possible catastrophes dominate. Our classification demonstrates that the field of AI risks is diverse, and includes many scenarios beyond the commonly discussed cases of a paperclip maximizer or robot-caused unemployment. Global catastrophic failure could happen at various levels of AI development, namely, before it starts self-improvement, during its takeoff, when it uses various instruments to escape its initial confinement, or after it successfully takes over the world and starts to implement its goal system, which could be plainly unaligned, or feature-flawed friendliness. AI could also halt at later stages of its development either due to technical glitches or ontological problems. Overall, we identified around several dozen scenarios of AI-driven global catastrophe. The extent of this list illustrates that there is no one simple solution to the problem of AI safety, and that AI safety theory is complex and must be customized for each AI development level.