Eliezer Yudkowsky | |
|---|---|
.jpg) Yudkowsky atStanford University in 2006 | |
| Born | Eliezer Shlomo (or Solomon) Yudkowsky (1979-09-11)September 11, 1979 (age 46) Chicago, Illinois |
| Organization | Machine Intelligence Research Institute |
| Known for | Coining the termfriendly artificial intelligence Research onAI safety Rationality writing Founder ofLessWrong |
| Website | www |
Eliezer S. Yudkowsky (/ˌɛliˈɛzərjʊdˈkaʊski/EL-ee-EH-zər yuud-KOW-skee;[1] born September 11, 1979) is an American artificial intelligence researcher[2][3][4][5] and writer ondecision theory andethics, best known for popularizing ideas related tofriendly artificial intelligence.[6][7] He is the founder of and a research fellow at theMachine Intelligence Research Institute (MIRI), a private research nonprofit based inBerkeley, California.[8] His work on the prospect of a runawayintelligence explosion influenced philosopherNick Bostrom's 2014 bookSuperintelligence: Paths, Dangers, Strategies.[9]
Yudkowsky's views on the safety challenges future generations of AI systems pose are discussed inStuart Russell's andPeter Norvig's undergraduate textbookArtificial Intelligence: A Modern Approach. Noting the difficulty of formally specifying general-purpose goals by hand, Russell and Norvig cite Yudkowsky's proposal that autonomous and adaptive systems be designed to learn correct behavior over time:
Yudkowsky (2008)[10] goes into more detail about how to design aFriendly AI. He asserts that friendliness (a desire not to harm humans) should be designed in from the start, but that the designers should recognize both that their own designs may be flawed, and that the robot will learn and evolve over time. Thus the challenge is one of mechanism design—to design a mechanism for evolving AI under a system of checks and balances, and to give the systems utility functions that will remain friendly in the face of such changes.[6]
In response to theinstrumental convergence concern, which implies that autonomous decision-making systems with poorly designed goals would have default incentives to mistreat humans, Yudkowsky and other MIRI researchers have recommended that work be done to specify software agents that converge on safe default behaviors even when their goals are misspecified.[11][7] Yudkowsky also proposed in 2004 a theoreticalAI alignment framework calledcoherent extrapolated volition, which involves designing AIs to pursue what people would desire under idealepistemic and moral conditions.[12]
In theintelligence explosion scenario hypothesized byI. J. Good, recursively self-improving AI systems quickly transition from subhuman general intelligence tosuperintelligence.Nick Bostrom's 2014 bookSuperintelligence: Paths, Dangers, Strategies sketches out Good's argument in detail, while citing Yudkowsky on the risk thatanthropomorphizing advanced AI systems will cause people to misunderstand the nature of an intelligence explosion. "AI might make anapparently sharp jump in intelligence purely as the result of anthropomorphism, the human tendency to think of 'village idiot' and 'Einstein' as the extreme ends of the intelligence scale, instead of nearly indistinguishable points on the scale of minds-in-general."[6][10][12]
InArtificial Intelligence: A Modern Approach, Russell and Norvig raise the objection that there are known limits to intelligent problem-solving fromcomputational complexity theory; if there are strong limits on how efficiently algorithms can solve various tasks, an intelligence explosion may not be possible.[6]
In a 2023 op-ed forTime magazine, Yudkowsky discussed the risk of artificial intelligence and advocated for international agreements to limit it, including a total halt on the development of AI.[13][14] He suggested that participating countries should be willing to take military action, such as "destroy[ing] a rogue datacenter by airstrike", to enforce such a moratorium.[5] The article helped introduce the debate about AI alignment to the mainstream, leading a reporter to ask PresidentJoe Biden a question about AI safety at a press briefing.[2]
Together withNate Soares, Yudkowsky wroteIf Anyone Builds It, Everyone Dies, which was published byLittle, Brown and Company on September 16, 2025.[15]
Between 2006 and 2009, Yudkowsky andRobin Hanson were the principal contributors toOvercoming Bias, a cognitive and social science blog sponsored by theFuture of Humanity Institute of Oxford University. In February 2009, Yudkowsky foundedLessWrong, a "community blog devoted to refining the art of human rationality".[16][17]Overcoming Bias has since functioned as Hanson's personal blog.
Over 300 blog posts by Yudkowsky on philosophy and science (originally written onLessWrong andOvercoming Bias) were released as an ebook,Rationality: From AI to Zombies, by MIRI in 2015.[18] This book is also referred to as The Sequences.[19] MIRI has also publishedInadequate Equilibria, Yudkowsky's 2017 ebook on societal inefficiencies.[20]
Yudkowsky has also written several works of fiction. Hisfanfiction novelHarry Potter and the Methods of Rationality uses plot elements fromJ. K. Rowling'sHarry Potter series to illustrate topics in science and rationality.[16][21][22]
Yudkowsky is anautodidact[23] and did not attend high school or college.[24] He is Jewish and was raised as aModern Orthodox Jew, but is now secular.[25][26]
Eliezer Yudkowsky, a researcher at the Machine Intelligence Research Institute, in the Bay Area, has likened A.I.-safety recommendations to a fire-alarm system. A classic experiment found that, when smoky mist began filling a room containing multiple people, most didn't report it. They saw others remaining stoic and downplayed the danger. An official alarm may signal that it's legitimate to take action. But, in A.I., there's no one with the clear authority to sound such an alarm, and people will always disagree about which advances count as evidence of a conflagration. "There will be no fire alarm that is not an actual running AGI," Yudkowsky has written. Even if everyone agrees on the threat, no company or country will want to pause on its own, for fear of being passed by competitors. ... That may require quitting A.I. cold turkey before we feel it's time to stop, rather than getting closer and closer to the edge, tempting fate. But shutting it all down would call for draconian measures—perhaps even steps as extreme as those espoused by Yudkowsky, who recently wrote, in an editorial forTime, that we should "be willing to destroy a rogue datacenter by airstrike," even at the risk of sparking "a full nuclear exchange."
The Sequences, the ur-text that gave rise to the [Rationalist] movement.
| International | |
|---|---|
| National | |
| Academics | |
| Artists | |
| Other | |
