Existential Risk

Research literature on threats to humanity's future

   

MAIN INTRODUCTORY PAPER

     
Existential Risk Prevention as the Most Important Task for Humanity. Nick Bostrom NEW pdf html
(2011) Working paper (revised) (Risk Analysis, under review)
ABSTRACT
Existential risks are those that threaten the entire future of humanity.  Many theories of value imply that even relatively small reductions in net existential risk have enormous expected value.  Despite their importance, issues surrounding human-extinction risks and related hazards remain poorly understood.  In this paper, I clarify the concept of existential risk and develop an improved classification scheme.  I discuss the relation between existential risks and basic issues in axiology, and show how existential risk reduction (via the maxipok rule) can serve as a strongly action-guiding principle for utilitarian concerns.  I also show how the notion of existential risk suggests a new way of thinking about the ideal of sustainability.
         
         
     

ORIGINAL PAPER

 
     
Existential Risks: Analyzing Human Extinction Scenarios and Related Hazard. Nick Bostrom pdf html
Journal of Evolution and Technology, Vol. 9, No. 1 (2002)
ABSTRACT
Because of accelerating technological progress, humankind may be rapidly approaching a critical phase in its career. In addition to well-known threats such as nuclear holocaust, the prospects of radically transforming technologies like nanotech systems and machine intelligence present us with unprecedented opportunities and risks. Our future, and whether we will have a future at all, may well be determined by how we deal with these challenges. In the case of radically transforming technologies, a better understanding of the transition dynamics from a human to a "posthuman" society is needed. Of particular importance is to know where the pitfalls are: the ways in which things could go terminally wrong. While we have had long exposure to various personal, local, and endurable global hazards, this paper analyzes a recently emerging category: that of existential risks. These are threats that could cause our extinction or destroy the potential of Earth-originating intelligent life. Some of these threats are relatively well known while others, including some of the gravest, have gone almost unrecognized. Existential risks have a cluster of features that make ordinary risk management ineffective. A final section of this paper discusses several ethical and policy implications. A clearer understanding of the threat picture will enable us to formulate better strategies. (This is the original paper that introduced the concept.)
Translations: Russian, Belorussian
             
       
     

FAQ

   
     
Existential Risk FAQ. Nick Bostrom NEW pdf html
(2012) Version 1.1
Short answers to 11 common questions
   
           
       
     

BACKGROUND READINGS AND RELATED WORKS

   
     

OTHER GENERAL DISCUSSION ON EXISTENTIAL RISK AND HUMAN EXTINCTION

Reducing the Risk of Human Extinction. Jason Gaverick Matheny. Risk Analysis, Vol. 27, No. 5 (2007): 1335-1344
— review article, including case for cost effectiveness of xrisk mitigation

Book:

John Leslie: The End of the World: The Science and Ethics of Human Extinction (New York: Routledge, 1996)
— focuses mainly on the doomsday arguments (see "observation selection theory" below), but also discusses some ethical and empirical issues

GLOBAL CATASTROPHIC RISKS

Introduction to Global Catastrophic Risks. Nick Bostrom & Milan Cirkovic. In Global Catastrophic Risks (Oxford: Oxford University Press, 2008): pp. 1-30
— GCRs is a superset of existential risks that also includes far less serious risks that would nevertheless have globally significant impacts. This is a book chapter, giving a broad overview of global catastrophic risks generally

Global Catastrophic Risks Survey. Anders Sandberg & Nick Bostrom. Future of Humanity Institute, Oxford University, Tech. Report (2008)/1
— result of an informal expert poll - the median probability to human extinction by 2100 was 19%

Literature on GCR
— reading list on global catastrophic risks from circa 2008

Some books on global catastrophic risks:

Nick Bostrom & Milan Cirkovic: Global Catastrophic Risks (Oxford: Oxford University Press, 2008)
Richard Posner: Catastrophe: Risk and Response (Oxford: Oxford University Press, 2004)
Jared Diamond: Collapse: Collapse: How Societies Choose to Fail or Succeed (Viking Adult, 2004)
Tad Homer-Dixon: The Upside of Down: Catastrophe, Creativity, and the Renewal of Civilization (Island Press, 2008)
Vaclav Smil: Global Catastrophes and Trends: The Next Fifty Years (Cambridge, MA: The MIT Press, 2008)

POPULAR

Book:

Martin Rees: Our Final Century (London, William Heineman, 2003)
— discusses extinction risks among other things, but note that Rees's oft-cited claim that there is a "fifty-fifty" chance that this is our last century refers to the likelihood that our "present civilization" will survive (yet its failure to do so does not necessarily imply human extinction or an existential catastrophe).

Dinosaurs, Dodos, Humans? Nick Bostrom. Global Agenda, January (2006): 230-231
— one popular summary

A Primer on the Doomsday Argument. Nick Bostrom. ehpilosopher, 12 May 2007
— a popular short introduction to one argument that claims the probability of existential risk has been systematically and radically underestimated

FERMI PARADOX (GREAT FILTER)

Where Are They? Why I Hope the Search for Extraterrestrial Life Finds Nothing. Nick Bostrom. MIT Technology Review, May/June issue (2008): 72-77
— the "Fermi paradox" is the lack of observation of any extraterrestrial life; this paper explores connection to existential risk

VALUE THEORY

Astronomical Waste: The Opportunity Cost of Delayed Technological Development. Nick Bostrom. Utilitas, Vol. 15, No. 3 (2003): 308-314
— argues that xrisk reduction should be a dominating concern for many consequentialists

On Becoming Extinct. James Lenman. Pacific Philosophical Quarterly, Vol. 83 (2002): 253-296 [link: may require journal subscription]
— argues that it doesn't matter when humanity goes extinct, provided it doesn't happen very soon

Fat-Tail Uncertainty in the Economics of Catastrophic Climate Change. Martin L. Weitzman. Preprint (Harvard, 2011)
— claims most expected harm from climate change is in the tail (of low-probability, high-consequence outcomes)

The Most Important Thing About Climate Change. John Broome. in Public Policy: Why Ethics Matters, ed. Jonathan Boston, Andrew Bradstock, & David Eng, ANU E Press (2010): 101-116
— critiques Weitzman's paper

OBSERVATION SELECTION THEORY AND OTHER METHODOLOGY

Anthropic Shadow: Observation Selection Effects and Human Extinction Risks. Milan Cirkovic, Anders Sandberg, & Nick Bostrom. Risk Analysis, Vol. 30, No. 10 (2010): 1495-1506
— illustrates how observation selection effects can bias inferences about some xrisk probabilities

How Unlikely is a Doomsday Catastrophe? Max Tegmark & Nick Bostrom. Nature, Vol. 438 (2005): 754
— earlier, more compact discussion of the above

Anthropic Bias: Observation Selection Effects in Science and Philosophy. Nick Bostrom. (New York: Routledge, 2002)
— book on anthropic reasoning, including the doomsday argument and other xrisk-relevant applications

Probing the Improbable: Methodological Challenges for Risks with Low Probabilities and High Stakes. Toby Ord, Rafaela Hillerbrand, & Anders Sandberg. Journal of Risk Research, Vol. 13, No. 2 (2010): 191-205
— when an analysis says a risk is extremely small, most net risk can lie in the possibility that the analysis is wrong

Book:

John Leslie: The End of the World: The Science and Ethics of Human Extinction. (New York: Routledge, 1996)
— Leslie's book emphasizes the controversial but potentially important Carter-Leslie doomsday argument, though his book also discusses some empirical issues relevant to assessing the risk of human extinction

   
         
 

  PIFT Oxford University