Nick Bostrom | |
|---|---|
 Bostrom in 2020 | |
| Born | Niklas Boström (1973-03-10)10 March 1973 (age 52) Helsingborg, Sweden |
| Spouse | Susan[2] |
| Education | |
| Education | |
| Thesis | Observational Selection Effects and Probability (2000) |
| Doctoral advisor | Colin Howson Craig Callender[1] |
| Philosophical work | |
| Era | Contemporary philosophy |
| Region | Western philosophy |
| School | Analytic philosophy[2] |
| Institutions | Yale University University of Oxford Future of Humanity Institute |
| Main interests | Philosophy of artificial intelligence Bioethics |
| Notable ideas | Anthropic bias Reversal test Simulation hypothesis Vulnerable world hypothesis Existential risk studies Singleton Ancestor simulation Information hazard Infinitarian paralysis[3] Self-indication assumption Self-sampling assumption |
| Website | nickbostrom |
Nick Bostrom (/ˈbɒstrəm/BOST-rəm;Swedish:Niklas Boström[ˈnɪ̌kːlasˈbûːstrœm]; born 10 March 1973)[4] is aphilosopher known for his work onexistential risk, theanthropic principle,human enhancement ethics,whole brain emulation,superintelligence risks, and thereversal test. He was the founding director of the now dissolvedFuture of Humanity Institute at theUniversity of Oxford[5] and is now Principal Researcher at the Macrostrategy Research Initiative.[6]
Bostrom is the author ofAnthropic Bias: Observation Selection Effects in Science and Philosophy (2002),Superintelligence: Paths, Dangers, Strategies (2014) andDeep Utopia: Life and Meaning in a Solved World (2024).
Bostrom believes that advances inartificial intelligence (AI) may lead tosuperintelligence, which he defines as "any intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest". He views this as a major source of opportunities and existential risks.[5][7]
Born as Niklas Boström in 1973 inHelsingborg,Sweden,[8] he disliked school at a young age and spent his last year of high school learning from home. He was interested in a wide variety of academic areas, including anthropology, art, literature, and science.[2]
He received aB.A. degree from theUniversity of Gothenburg in 1994.[9] He then earned anM.A. degree inphilosophy andphysics fromStockholm University and anMSc degree incomputational neuroscience fromKing's College London in 1996. During his time at Stockholm University, he researched the relationship between language and reality by studying the analytic philosopherW. V. Quine.[2] He also did some turns on London's stand-up comedy circuit.[8] In 2000, he was awarded aPhD degree in philosophy from theLondon School of Economics under the supervision ofColin Howson andCraig Callender.[1] His thesis was titledObservational selection effects and probability.[1] He held a teaching position atYale University from 2000 to 2002, and was aBritish Academy Postdoctoral Fellow at theUniversity of Oxford from 2002 to 2005.[10]
Bostrom's research concerns the future of humanity and long-term outcomes.[5][11] He discussesexistential risk,[2] which he defines as one in which an "adverse outcome would either annihilate Earth-originating intelligent life or permanently and drastically curtail its potential". Bostrom is mostly concerned about anthropogenic risks, which are risks arising from human activities, particularly from new technologies such as advanced artificial intelligence,molecular nanotechnology, orsynthetic biology.[12]
In 2005, Bostrom founded theFuture of Humanity Institute which,[2] until its shutdown in 2024, researched thefar future of human civilization.[13][14] He is also an adviser to theCentre for the Study of Existential Risk.[11]
In the 2008 essay collection,Global Catastrophic Risks, editors Bostrom andMilan M. Ćirković characterize the relationship between existential risk and the broader class of global catastrophic risks, and link existential risk toobserver selection effects[15] and theFermi paradox.[16]
In a paper called "TheVulnerable World Hypothesis",[17] Bostrom suggests that there may be some technologies that destroy human civilization by default[a] when discovered. Bostrom proposes a framework for classifying and dealing with these vulnerabilities. He also gives counterfactual thought experiments of how such vulnerabilities could have historically occurred, e.g. if nuclear weapons had been easier to develop or had ignited the atmosphere (asEdward Teller had feared).[19]
Bostrom supports the substrate independence principle, the idea that consciousness can emerge on various types of physical substrates, not only in "carbon-based biological neural networks" like the human brain.[20] He considers that "sentience is a matter of degree"[21] and that digital minds can in theory be engineered to have a much higher rate and intensity ofsubjective experience than humans, using less resources. Such highly sentient machines, which he calls "super-beneficiaries", would be extremely efficient at achieving happiness. He recommends finding "paths that will enable digital minds and biological minds to coexist, in a mutually beneficial way where all of these different forms can flourish and thrive".[22]
Bostrom has published numerous articles onanthropic reasoning, as well as the bookAnthropic Bias: Observation Selection Effects in Science and Philosophy. In the book, he criticizes previous formulations of the anthropic principle, including those ofBrandon Carter,John Leslie,John Barrow, andFrank Tipler.[23]
Bostrom believes that the mishandling ofindexical information is a common flaw in many areas of inquiry (including cosmology, philosophy, evolution theory, game theory, and quantum physics). He argues that an anthropic theory is needed to deal with these. He introduces theself-sampling assumption (SSA) and analyzes theself-indication assumption (SIA), shows how they lead to different conclusions in a number of cases, and identifies how each is affected by paradoxes or counterintuitive implications in certain thought experiments. He argues against SIA and proposes refining SSA into the strong self-sampling assumption (SSSA), which replaces "observers" in the SSA definition with "observer-moments".[24]
In later work, he proposed withMilan M. Ćirković andAnders Sandberg the phenomenon ofanthropic shadow, an observation selection effect that prevents observers from observing certain kinds of catastrophes in their recent geological and evolutionary past. They suggest that events that lie in the anthropic shadow are likely to be underestimated unless statistical corrections are made.[25][26]
Bostrom's simulation argument posits that at least one of the following statements is very likely to be true:[27][28]
Bostrom is favorably disposed toward "human enhancement", or "self-improvement and human perfectibility through the ethical application of science", as well as a critic of bio-conservative views.[29]
In 1998, Bostrom co-founded (withDavid Pearce) the WorldTranshumanist Association[29] (which has since changed its name toHumanity+). In 2004, he co-founded (withJames Hughes) theInstitute for Ethics and Emerging Technologies,[30] although he is no longer involved with either of these organisations.
In 2005, Bostrompublished the short story "The Fable of the Dragon-Tyrant" in theJournal of Medical Ethics. A shorter version was published in 2012 inPhilosophy Now.[31] The fable personifies death as a dragon that demands a tribute of thousands of people every day. The story explores howstatus quo bias and learned helplessness can prevent people from taking action to defeat aging even when the means to do so are at their disposal. YouTuberCGP Grey created an animated version of the story.[32]
With philosopherToby Ord, he proposed thereversal test in 2006. Given humans' irrational status quo bias, how can one distinguish between valid criticisms of proposed changes in a human trait and criticisms merely motivated by resistance to change? The reversal test attempts to do this by asking whether it would be a good thing if the trait was altered in the opposite direction.[33]
Bostrom's work also considers potentialdysgenic effects in human populations but he thinksgenetic engineering can provide a solution and that "In any case, the time-scale for human natural genetic evolution seems much too grand for such developments to have any significant effect before other developments will have made the issue moot".[34]
Bostrom has suggested that technology policy aimed at reducing existential risk should seek to influence the order in which various technological capabilities are attained, proposing the principle ofdifferential technological development. This principle states that we ought to retard the development of dangerous technologies, particularly ones that raise the level of existential risk, and accelerate the development of beneficial technologies, particularly those that protect against the existential risks posed by nature or by other technologies.[34]
In 2011, Bostrom founded the Oxford Martin Program on the Impacts of Future Technology.[35]
Bostrom's theory of the unilateralist's curse has been cited as a reason for the scientific community to avoid controversial dangerous research such as reanimating pathogens.[36]
In 2014, Bostrom publishedSuperintelligence: Paths, Dangers, Strategies, which became aNew York Times Best Seller.[37]The book argues thatsuperintelligence is possible and explores different types of superintelligences, their cognition, the associated risks. He also presents technical and strategic considerations on how to make it safe.
Bostrom explores multiple possible paths to superintelligence, includingwhole brain emulation and human intelligence enhancement, but focuses onartificial general intelligence, explaining that electronic devices havemany advantages over biological brains.[38]
Bostrom draws a distinction betweenfinal goals and instrumental goals. A final goal is what anagent tries to achieve for its own intrinsic value. Instrumental goals are just intermediary steps towards final goals. Bostrom contends there are instrumental goals that will be shared by most sufficiently intelligent agents because they are generally useful to achieve any objective (e.g. preserving the agent's own existence or current goals, acquiring resources, improving its cognition...), this is the concept ofinstrumental convergence. On the other side, he writes that virtually any level of intelligence can in theory be combined with virtually any final goal (even absurd final goals, e.g.making paperclips), a concept he calls theorthogonality thesis.[39]
He argues that an AI with the ability to improve itself might initiate anintelligence explosion, resulting (potentially rapidly) in a superintelligence.[40] Such a superintelligence could have vastly superior capabilities, notably in strategizing, social manipulation, hacking or economic productivity. With such capabilities, a superintelligence could outwit humans and take over the world, establishing asingleton (which is "a world order in which there is at the global level a single decision-making agency"[b]) and optimizing the world according to its final goals.[38]
Bostrom argues that giving simplistic final goals to a superintelligence could be catastrophic:
Suppose we give an A.I. the goal to make humans smile. When the A.I. is weak, it performs useful or amusing actions that cause its user to smile. When the A.I. becomes superintelligent, it realizes that there is a more effective way to achieve this goal: take control of the world and stick electrodes into the facial muscles of humans to cause constant, beaming grins.[34]
Bostrom explores several pathways to reduce theexistential risk from AI. He emphasizes the importance of international collaboration, notably to reducerace to the bottom andAI arms race dynamics. He suggests potential techniques to help control AI, including containment, stunting AI capabilities or knowledge, narrowing the operating context (e.g. to question-answering), or "tripwires" (diagnostic mechanisms that can lead to a shutdown).[38] But Bostrom contends that "we should not be confident in our ability to keep a superintelligent genie locked up in its bottle forever. Sooner or later, it will out". He thus suggests that in order to be safe for humanity, superintelligence must bealigned with morality or human values so that it is "fundamentally on our side".[41] Potential AInormativity frameworks includeYudkowsky'scoherent extrapolated volition (human values improved via extrapolation),moral rightness (doing what is morally right), and moral permissibility (following humanity's coherent extrapolated volition except when it's morally impermissible).[38]
Bostrom warns that an existential catastrophe can also occur from AI being misused by humans for destructive purposes, or from humans failing to take into account the potential moral status of digital minds. Despite these risks, he says that machine superintelligence seems involved at some point in "all the plausible paths to a really great future".[7]
The book became aNew York Times Best Seller and received positive feedback from personalities such asStephen Hawking,Bill Gates,Elon Musk,Peter Singer andDerek Parfit. It was praised for offering clear and compelling arguments on a neglected yet important topic. It was sometimes criticized for spreading pessimism about the potential of AI, or for focusing on longterm and speculative risks.[42] Some skeptics such asDaniel Dennett orOren Etzioni contended that superintelligence is too far away for the risk to be significant.[43][44]Yann LeCun considers that there is no existential risk, asserting that superintelligent AI will have no desire for self-preservation[45] and that experts can be trusted to make it safe.[46]
Raffi Khatchadourian wrote that Bostrom's book on superintelligence "is not intended as a treatise of deep originality; Bostrom's contribution is to impose the rigors ofanalytic philosophy on a messy corpus of ideas that emerged at the margins of academic thought."[42]
In his 2024 book,Deep Utopia: Life and Meaning in a Solved World, Bostrom explores the concept of an ideal life, if humanity transitions successfully into a post-superintelligence world. Bostrom notes that the question is "not how interesting a future is to look at, but how good it is to live in." He outlines some technologies that he considers physically possible in theory and available at technological maturity, such ascognitive enhancement,reversal of aging, arbitrary sensory inputs (taste, sound...), or the precise control of motivation, mood, well-being and personality. According to him, not only machines would be better than humans at working, but they would also undermine the purpose of many leisure activities, providing extreme welfare while challenging the quest for meaning.[47][48]
Bostrom has provided policy advice and consulted for many governments and organizations. He gave evidence to theHouse of Lords, Select Committee on Digital Skills.[49] He is an advisory board member for theMachine Intelligence Research Institute,[50]Future of Life Institute,[51] and an external advisor for the CambridgeCentre for the Study of Existential Risk.[52]
In January 2023, Bostrom issued an apology[53] for a 1996listserv email[54] he sent as a postgrad where he had stated that he thought "Blacks are more stupid than whites", and where he also used the word "niggers" in a description of how he thought this statement might be perceived by others.[55][56] The apology, posted on his website,[53] stated that "the invocation of a racial slur was repulsive" and that he "completely repudiate[d] this disgusting email".[55]
The email has been described as "racist" in several news sources.[57][58][59][55] According toAndrew Anthony ofThe Guardian, "The apology did little to placate Bostrom’s critics, not least because he conspicuously failed to withdraw his central contention regardingrace and intelligence, and seemed to make a partial defence ofeugenics."[60]
Shortly afterward, Oxford University condemned the language used in the email and started an investigation.[55][58] The investigation concluded on 10 August 2023: "[W]e do not consider you to be a racist or that you hold racist views, and we consider that the apology you posted in January 2023 was sincere."[61][60]
Bostrom met his wife Susan in 2002. As of 2015, she lived inMontreal and Bostrom in Oxford. They have one son.[2]
Nick Bostrom posted a note on his website apologizing for the appallingly racist listserv email.
The closure of Bostrom's center is a further blow to the effective altruism and long-termism movements that the philosopher had spent decades championing, and which in recent years have become mired in scandals related to racism, sexual harassment and financial fraud. Bostrom himself issued an apology last year after a decades-old email surfaced in which he claimed "Blacks are more stupid than whites" and used the N-word.
Influential Oxford philosopher Nick Bostrom, well-known for his work on philosophical questions related to ethics, the future, and technology (existential risk, artificial intelligence, simulation), posted an apology for a blatantly racist email he sent to a listserv 26 years ago.
Nick Bostrom, an influential philosopher at the University of Oxford who has been called the "father" of the longtermism movement, has apologized for a racist email he sent in the mid-90s. In the email, Bostrom said that "Blacks are more stupid than whites," adding, "I like that sentence and think it is true," and used a racial slur.
The apology did little to placate Bostrom's critics, not least because he conspicuously failed to withdraw his central contention regarding race and intelligence, and seemed to make a partial defence of eugenics. Although, after an investigation, Oxford University did accept that Bostrom was not a racist, the whole episode left a stain on the institute's reputation
we do not consider you to be a racist or that you hold racist views, and we consider that the apology you posted in January 2023 was sincere. … we believe that your apology, your acknowledgement of the distress your actions caused, and your appreciation for the care and time that everyone has given to this process has been genuine and sincere. We were also encouraged that you have already embarked on a journey of deep and meaningful reflection, which includes exploring the learning and self-education from this process.
| International | |
|---|---|
| National | |
| Academics | |
| Artists | |
| People | |
| Other | |
