_book_cover.jpg) Hardcover edition | |
| Author | Stuart J. Russell |
|---|---|
| Language | English |
| Subject | AI control problem |
| Genre | Non-fiction |
| Publisher | Viking |
Publication date | October 8, 2019 |
| Publication place | United States |
| Pages | 352 |
| ISBN | 978-0-525-55861-3 |
| OCLC | 1083694322 |
Human Compatible: Artificial Intelligence and the Problem of Control is a 2019 non-fiction book by computer scientistStuart J. Russell. It asserts that therisk to humanity from advancedartificial intelligence (AI) is a serious concern despite the uncertainty surrounding future progress in AI. It also proposes an approach to theAI control problem.
Russell begins by asserting that the standard model of AI research, in which the primary definition of success is getting better and better at achieving rigid human-specified goals, is dangerously misguided. Such goals may not reflect what human designers intend, such as by failing to take into account any human values not included in the goals. If an AI developed according to the standard model were to becomesuperintelligent, it would likely not fully reflect human values and could be catastrophic to humanity. Russell asserts that precisely because the timeline for developing human-level or superintelligent AI is highly uncertain, safety research should be begun as soon as possible, as it is also highly uncertain how long it would take to complete such research.
Russell argues that continuing progress in AI capability is inevitable because of economic pressures. Such pressures can already be seen in the development of existing AI technologies such asself-driving cars andpersonal assistant software. Moreover, human-level AI could be worth many trillions of dollars. Russell then examines the current debate surrounding AI risk. He offers refutations to a number of common arguments dismissing AI risk and attributes much of their persistence to tribalism—AI researchers may see AI risk concerns as an "attack" on their field. Russell reiterates that there are legitimate reasons to take AI risk concerns seriously and that economic pressures make continued innovation in AI inevitable.
Russell then proposesan approach to developing provably beneficial machines that focus on deference to humans. Unlike in the standard model of AI, where the objective is rigid and certain, this approach would have the AI's true objective remain uncertain, with the AI only approaching certainty about it as it gains more information about humans and the world. This uncertainty would, ideally, prevent catastrophic misunderstandings of human preferences and encourage cooperation and communication with humans. Russell concludes by calling for tighter governance of AI research and development as well as cultural introspection about the appropriate amount of autonomy to retain in an AI-dominated world.
Russell lists three principles to guide the development of beneficial machines. He emphasizes that these principles are not meant to be explicitly coded into the machines; rather, they are intended for human developers. The principles are as follows:[1]: 173
1. The machine's only objective is to maximize the realization of human preferences.
2. The machine is initially uncertain about what those preferences are.
3. The ultimate source of information about human preferences is human behavior.
The "preferences" Russell refers to "are all-encompassing; they cover everything you might care about, arbitrarily far into the future."[1]: 173 Similarly, "behavior" includes any choice between options,[1]: 177 and the uncertainty is such that some probability, which may be quite small, must be assigned to every logically possible human preference.[1]: 201
Russell exploresinverse reinforcement learning, in which a machine infers a reward function from observed behavior, as a possible basis for a mechanism for learning human preferences.[1]: 191–193
Several reviewers agreed with the book's arguments. Ian Sample inThe Guardian called it "convincing" and "the most important book on AI this year".[2] Richard Waters of theFinancial Times praised the book's "bracing intellectual rigour".[3]Kirkus Reviews endorsed it as "a strong case for planning for the day when machines can outsmart us".[4]
The same reviewers characterized the book as "wry and witty",[2] or "accessible"[4] due to its "laconic style and dry humour".[3] Matthew Hutson of theWall Street Journal said "Mr. Russell's exciting book goes deep while sparkling with dry witticisms".[5] ALibrary Journal reviewer called it "The right guide at the right time".[6]
James McConnachie ofThe Times wrote "This is not quite the popular book that AI urgently needs. Its technical parts are too difficult, and its philosophical ones too easy. But it is fascinating and significant."[7]
By contrast,Human Compatible was criticized in itsNature review by David Leslie, an Ethics Fellow at theAlan Turing Institute; and similarly in aNew York Times opinion essay byMelanie Mitchell. One point of contention was whethersuperintelligence is possible. Leslie states Russell "fails to convince that we will ever see the arrival of a 'second intelligent species'",[8] and Mitchell doubts a machine could ever "surpass the generality and flexibility of human intelligence" without losing "the speed, precision, and programmability of a computer".[9] A second disagreement was whether intelligent machines would naturally tend to adopt so-called "common sense" moral values. In Russell's thought experiment about ageoengineering robot that "asphyxiates humanity to deacidify the oceans", Leslie "struggles to identify any intelligence". Similarly, Mitchell believes an intelligent robot would naturally tend to be "tempered by the common sense, values and social judgment without which general intelligence cannot exist".[10][11]
The book was longlisted for the 2019Financial Times/McKinsey Award.[12]
