Nick Bostrom

The ethical issues related to the possible future creation of machines with general intellectual capabilities far outstripping those of humans are quite distinct from any ethical problems arising in current automation and information systems. Such superintelligence would not be just another technological development; it would be the most important invention ever made, and would lead to explosive progress in all scientific and technological fields, as the superintelligence would conduct research with superhuman efficiency. To the extent that ethics is a cognitive pursuit, a superintelligence could also easily surpass humans in the quality of its moral thinking. However, it would be up to the designers of the superintelligence to specify its original motivations. Since the superintelligence may become unstoppably powerful because of its intellectual superiority and the technologies it could develop, it is crucial that it be provided with human-friendly motivations. This paper surveys some of the unique ethical issues in creating superintelligence, and discusses what motivations we ought to give a superintelligence, and introduces some cost-benefit considerations relating to whether the development of superintelligent machines ought to be accelerated or retarded.

_This paper outlines the case for believing that we will have superhuman artificial intelligence_ _within the first third of the next century. It looks at different estimates of the processing power of_ _the human brain; how long it will take until computer hardware achieve a similar performance;_ _ways of creating the software through bottom-up approaches like the one used by biological_ _brains; how difficult it will be for neuroscience figure out enough about how brains work to_ _make this approach work; and how fast we can expect superintelligence to be developed once_ _there is human-level artificial intelligence._.

We differentiate morally between actual and potential beings: the latter do not exist now and will never exist unless we bring them into existence. The interests of existing persons should guide the creation of new beings. We ought not to create new beings that are expected to harm the interests of existing persons. If a potential being becomes actual, it becomes a member of the moral community and its interests should be taken into account. A being can be actual even if it does not currently exist: there is a moral reason not to set up a mechanism today that will harm a being that comes into existence tomorrow.

The idea of life on Mars has been with us for nearly 300 years, ever since early astronomers saw what they believed to be polar ice caps through their primitive telescopes. Since then, space probes have indeed confirmed that the red planet has water and future missions might tell us if Mars contains any traces of life, whether extinct or still active. Such a discovery would be of tremendous scientific significance: the first time that any signs of extraterrestrial life have ever been detected. Many people would also find it heartening to learn that we’re not entirely alone in this vast cold cosmos.

Human enhancement has emerged in recent years as a blossoming topic in applied ethics. With continuing advances in science and technology, people are beginning to realize that some of the basic parameters of the human condition might be changed in the future. One important way in which the human condition could be changed is through the enhancement of basic human capacities. If this becomes feasible within the lifespan of many people alive today, then it is important now to consider the normative questions raised by such prospects. The answers to these questions might not only help us be better prepared when technology catches up with imagination, but they may be relevant to many decisions we make today, such as decisions about how much funding to give to various kinds of research. Enhancement is typically contraposed to therapy. In broad terms, therapy aims to fix something that has gone wrong, by curing specific diseases or injuries, while enhancement interventions aim to improve the state of an organism beyond its normal healthy state. However, the distinction between therapy and enhancement is problematic, for several reasons. First, we may note that the therapy-enhancement dichotomy does not map onto any corresponding dichotomy between standard-contemporary-medicine and medicineas-it-could-be-practised-in-the-future. Standard contemporary medicine includes many practices that do not aim to cure diseases or injuries. It includes, for example, preventive medicine, palliative care, obstetrics, sports medicine, plastic surgery, contraceptive devices, fertility treatments, cosmetic dental procedures, and much else. At the same time, many enhancement interventions occur outside of the medical framework. Office workers enhance their performance by drinking coffee. Make-up and grooming are used to enhance appearance. Exercise, meditation, fish oil, and St John’s Wort are used to enhance mood. Second, it is unclear how to classify interventions that reduce the probability of disease and death..

Lang, lang geleden werd de planeet getiranniseerd door een gigantische draak. De draak was groter dan de grootste kerk en was bedekt met dikke, zwarte schubben. Zijn rode ogen gloeiden van haat en uit zijn verschrikkelijke bek stroomde onophoudelijk stinkend, geelgroen slijm. Hij eiste van de mens een ijzingwekkend eerbetoon: om zijn enorme honger te stillen, moesten er iedere avond bij het vallen van het duister tienduizend mannen en vrouwen aan de voet van de berg, waar de draak-tiran woonde, afgeleverd worden. Soms slokte de draak deze arme zielen gelijk op; soms sloot hij ze echter op in de berg waar zij maanden of jaren wegkwijnden totdat zij uiteindelijk opgegeten werden.

This paper argues that our current best cosmological theories, according to which cosmos is very big are compatible with all possible evidence. The problem is unrelated to the Quine-Duhem underdetermination thesis. The compatibility to which this paper draws attention is much more radical: it appears as if all of our best cosmological theories are perfectly probabilistically compatible with all possible evidence and that no empirical discovery could give us any reason whatever to favor one such theory over another. This consequence is absurd. In order to create an evidential link between cosmological theory and observation, a new methodological principle is needed. A candidate for such a principle is proposed, using a Bayesian framework that takes account of observation selection effects. Various applications in other scientific fields are considered.

The problem of cortical integration is described and various proposed solutions, including grandmother cells, cell assemblies, feed-forward structures, RAAM and synchronization, are reviewed. One method, involving complex attractors, that has received little attention in the literature, is explained and developed. I call this binding through annexation. A simulation study is then presented which suggests ways in which complex attractors could underlie our capacity to reason. The paper ends with a discussion of the efficiency and biological plausibility of the proposals as integration mechanisms for different regions and functions of the brain.

Does human enhancement threaten our dignity as some prominent commentators have asserted? Or could our dignity perhaps be technologically enhanced? After disentangling several different concepts of dignity, this essay focuses on the idea of dignity as a quality, a kind of excellence admitting of degrees and applicable to entities both within and without the human realm. I argue that dignity in this sense interacts with enhancement in complex ways which bring to light some fundamental issues in value theory, and that the effects of any given enhancement must be evaluated in its appropriate empirical context. Yet it is possible that through enhancement we could become better able to appreciate and secure many forms of dignity that are overlooked or missing under current conditions. I also suggest that in a posthuman world, dignity as a quality could grow in importance as an organizing moral/aesthetic idea.

George Sowers tries to refute the Doomsday argument on grounds that true random sampling requires all possible samples to be equally probable the time when the sample is taken. Yet the Doomsday argument does not rely on true random sampling. It presupposes random sampling only in a metaphorical sense. After arguing that Sowers’ critique fails, I outline my own view on the matter, which is that the Doomsday argument is inconclusive and that by developing a theory of observation selection effects one can show why that is so.

Because of accelerating technological progress, humankind may be rapidly approaching a critical phase in its career. In addition to well-known threats such as nuclear holocaust, the prospects of radically transforming technologies like nanotech systems and machine intelligence present us with unprecedented opportunities and risks. Our future, and whether we will have a future at all, may well be determined by how we deal with these challenges. In the case of radically transforming technologies, a better understanding of the transition dynamics from a human to a “posthuman” society is needed. Of particular importance is to know where the pitfalls are: the ways in which things could go terminally wrong. While we have had long exposure to various personal, local, and endurable global hazards, this paper analyzes a recently emerging category: that of existential risks. These are threats that could cause our extinction or destroy the potential of Earth-originating intelligent life. Some of these threats are relatively well known while others, including some of the gravest, have gone almost unrecognized. Existential risks have a cluster of features that make ordinary risk management ineffective. A final section of this paper discusses several ethical and policy implications. A clearer understanding of the threat picture will enable us to formulate better strategies.

_Anthropic Bias_ explores how to reason when you suspect that your evidence is biased by "observation selection effects"--that is, evidence that has been filtered by the precondition that there be some suitably positioned observer to "have" the evidence. This conundrum--sometimes alluded to as "the anthropic principle," "self-locating belief," or "indexical information"--turns out to be a surprisingly perplexing and intellectually stimulating challenge, one abounding with important implications for many areas in science and philosophy. There are the philosophical thought experiments and paradoxes: the Doomsday Argument; Sleeping Beauty; the Presumptuous Philosopher; Adam & Eve; the Absent-Minded Driver; the Shooting Room. And there are the applications in contemporary science: cosmology ("How many universes are there?", "Why does the universe appear fine-tuned for life?"); evolutionary theory ("How improbable was the evolution of intelligent life on our planet?"); the problem of time's arrow ("Can it be given a thermodynamic explanation?"); quantum physics ("How can the many-worlds theory be tested?"); game-theory problems with imperfect recall ("How to model them?"); even traffic analysis ("Why is the 'next lane' faster?"). _Anthropic Bias_ argues that the same principles are at work across all these domains. And it offers a synthesis: a mathematically explicit theory of observation selection effects that attempts to meet scientific needs while steering clear of philosophical paradox.

Blackballing the reaper is an old ambition, and considerable progress has been made. For the past 150 years, best-performance life-expectancy (i.e. life-expectancy in the country where it is highest) has increased at a very steady rate of 3 months per year.1 Lifeexpectancy for the ancient Romans was circa 23 years; today the average life-expectancy in the world is 64 years.2 Will this trend continue? What are the consequences if it does? And what ethical and political challenges does the prospect of life-extension create for us today? This article comments on some views on the ethics, science, and politics of lifeextension from a recent edited volume, The Fountain of Youth.

Когда-то Земля находилась под тиранией гигантского драконa. Ростом дракон был выше самого высокого собора, и весь был покрыт черной чешуей. Его красные глаза пылали ненавистью, и изо рта тёк непрерывный поток зловонной желтовато- зеленой слизи. Дракон требовал от человечества чудовищную дань: для удовлетворения его непомерного аппетита, десять тысяч мужчин и женщин должны были быть доставлены на исходе каждого дня к подножью горы у которой жил дракон-тиран. Некоторых дракон пожирал сразу, других держал в течение многих месяцев или даже лет перед тем как они были съедены.

With very advanced technology, a very large population of people living happy lives could be sustained in the accessible region of the universe. For every year that development of such technologies and colonization of the universe is delayed, there is therefore a corresponding opportunity cost: a potential good, lives worth living, is not being realized. Given some plausible assumptions, this cost is extremely large. However, the lesson for standard utilitarians is not that we ought to maximize the pace of technological development, but rather that we ought to maximize its safety, i.e. the probability that colonization will eventually occur. This goal has such high utility that standard utilitarians ought to focus all their efforts on it. Utilitarians of a ‘person-affecting’ stripe should accept a modified version of this conclusion. Some mixed ethical views, which combine utilitarian considerations with other criteria, will also be committed to a similar bottom line.