Part ofa series on
Artificial intelligence (AI)
Major goals Artificial general intelligence Intelligent agent Recursive self-improvement Planning Computer vision General game playing Knowledge representation Natural language processing Robotics AI safety
Approaches Machine learning Symbolic Deep learning Bayesian networks Evolutionary algorithms Hybrid intelligent systems Systems integration Open-source
Applications Bioinformatics Deepfake Earth sciences Finance Generative AI Art Audio Music Government Healthcare Mental health Industry Software development Translation Military Physics Projects
Philosophy AI alignment Artificial consciousness The bitter lesson Chinese room Friendly AI Ethics Existential risk Turing test Uncanny valley Human–AI interaction
History Timeline Progress AI winter AI boom AI bubble
Controversies Deepfake pornography Taylor Swift deepfake pornography controversy Google Gemini image generation controversy Pause Giant AI Experiments Removal of Sam Altman from OpenAI Statement on AI Risk Tay (chatbot) Théâtre D'opéra Spatial Voiceverse NFT plagiarism scandal
Glossary Glossary
v t e

AlphaGo Zero is a version ofDeepMind'sGo softwareAlphaGo. AlphaGo's team published an article inNature in October 2017 introducing AlphaGo Zero, a version created without using data from human games, and stronger than any previous version.^[1] By playing games against itself, AlphaGo Zero: surpassed the strength ofAlphaGo Lee in three days by winning 100 games to 0; reached the level ofAlphaGo Master in 21 days; and exceeded all previous versions in 40 days.^[2]

Trainingartificial intelligence (AI) withoutdatasets derived fromhuman experts has significant implications for the development of AI with superhuman skills, as expert data is "often expensive, unreliable, or simply unavailable."^[3]Demis Hassabis, the co-founder and CEO of DeepMind, said that AlphaGo Zero was so powerful because it was "no longer constrained by the limits of human knowledge".^[4] Furthermore, AlphaGo Zero performed better than standarddeep reinforcement learning models (such asDeep Q-Network implementations^[5]) due to its integration ofMonte Carlo tree search.David Silver, one of the first authors of DeepMind's papers published inNature on AlphaGo, said that it is possible to have generalized AI algorithms by removing the need to learn from humans.^[6]

Google later developedAlphaZero, a generalized version of AlphaGo Zero that could playchess andshōgi in addition to Go.^[7] In December 2017, AlphaZero beat the 3-day version of AlphaGo Zero by winning 60 games to 40, and with 8 hours of training it outperformed AlphaGo Lee on anElo scale. AlphaZero also defeated a top chess program (Stockfish) and a top Shōgi program (Elmo).^[8][9]

The network in AlphaGo Zero is aResNet with two heads.^{[1]: Appendix: Methods}

• The stem of the network takes as input a 17x19x19 tensor representation of the Go board.
  • 8 channels are the positions of the current player's stones from the last eight time steps. (1 if there is a stone, 0 otherwise. If the time step go before the beginning of the game, then 0 in all positions.)
  • 8 channels are the positions of the other player's stones from the last eight time steps.
  • 1 channel is all 1 if black is to move, and 0 otherwise.
• The body is aResNet with either 20 or 40 residual blocks and 256 channels.
• There are two heads, a policy head and a value head.
  • Policy head outputs alogit array of size${\displaystyle 19\times 19+1}$, representing the logit of making a move in one of the points, plus the logit ofpassing.
  • Value head outputs a number in the range${\displaystyle (-1,+1)}$, representing the expected score for the current player. -1 represents current player losing, and +1 winning.

AlphaGo Zero's neural network was trained usingTensorFlow, with 64 GPU workers and 19 CPU parameter servers.Only fourTPUs were used for inference. Theneural network initially knew nothing aboutGo beyond therules. Unlike earlier versions of AlphaGo, Zero only perceived the board's stones, rather than having some rare human-programmed edge cases to help recognize unusual Go board positions. The AI engaged inreinforcement learning, playing against itself until it could anticipate its own moves and how those moves would affect the game's outcome.^[10] In the first three days AlphaGo Zero played 4.9 million games against itself in quick succession.^[11] It appeared to develop the skills required to beat top humans within just a few days, whereas the earlier AlphaGo took months of training to achieve the same level.^[12]

According to Epoch.ai, training cost 3e23 FLOPs.^[13]

For comparison, the researchers also trained a version of AlphaGo Zero using human games, AlphaGo Master, and found that it learned more quickly, but actually performed more poorly in the long run.^[14] DeepMind submitted its initial findings in a paper toNature in April 2017, which was then published in October 2017.^[1]

The hardware cost for a single AlphaGo Zero system in 2017, including the four TPUs, has been quoted as around $25 million.^[15]

According to Hassabis, AlphaGo's algorithms are likely to be of the most benefit to domains that require an intelligent search through an enormous space of possibilities, such asprotein folding (seeAlphaFold) or accurately simulating chemical reactions.^[16] AlphaGo's techniques are probably less useful in domains that are difficult to simulate, such as learning how to drive a car.^[17] DeepMind stated in October 2017 that it had already started active work on attempting to use AlphaGo Zero technology for protein folding, and stated it would soon publish new findings.^[18][19]

AlphaGo Zero was widely regarded as a significant advance, even when compared with its groundbreaking predecessor, AlphaGo.Oren Etzioni of theAllen Institute for Artificial Intelligence called AlphaGo Zero "a very impressive technical result" in "both their ability to do it—and their ability to train the system in 40 days, on four TPUs".^[10]The Guardian called it a "major breakthrough for artificial intelligence", citing Eleni Vasilaki ofSheffield University and Tom Mitchell ofCarnegie Mellon University, who called it an impressive feat and an “outstanding engineering accomplishment" respectively.^[17]Mark Pesce of the University of Sydney called AlphaGo Zero "a big technological advance" taking us into "undiscovered territory".^[20]

Gary Marcus, a psychologist atNew York University, has cautioned that for all we know, AlphaGo may contain "implicit knowledge that the programmers have about how to construct machines to play problems like Go" and will need to be tested in other domains before being sure that its base architecture is effective at much more than playing Go. In contrast, DeepMind is "confident that this approach is generalisable to a large number of domains".^[11]

In response to the reports, South Korean Go professionalLee Sedol said, "The previous version of AlphaGo wasn’t perfect, and I believe that’s why AlphaGo Zero was made."On the potential for AlphaGo's development, Lee said he will have to wait and see but also said it will affect young Go players.Mok Jin-seok, who directs the South Korean national Go team, said the Go world has already been imitating the playing styles of previous versions of AlphaGo and creating new ideas from them, and he is hopeful that new ideas will come out from AlphaGo Zero. Mok also added that general trends in the Go world are now being influenced by AlphaGo's playing style. "At first, it was hard to understand and I almost felt like I was playing against an alien. However, having had a great amount of experience, I’ve become used to it," Mok said. "We are now past the point where we debate the gap between the capability of AlphaGo and humans. It’s now between computers."Mok has reportedly already begun analyzing the playing style of AlphaGo Zero along with players from the national team."Though having watched only a few matches, we received the impression that AlphaGo Zero plays more like a human than its predecessors," Mok said.^[21]Chinese Go professionalKe Jie commented on the remarkable accomplishments of the new program: "A pure self-learning AlphaGo is the strongest. Humans seem redundant in front of its self-improvement."^[22]

On 5 December 2017, DeepMind team released a preprint onarXiv, introducing AlphaZero, a program using generalized AlphaGo Zero's approach, which achieved within 24 hours a superhuman level of play inchess,shogi, andGo, defeating world-champion programs,Stockfish,Elmo, and 3-day version of AlphaGo Zero in each case.^[8]

AlphaZero (AZ) is a more generalized variant of the AlphaGo Zero (AGZ)algorithm, and is able to play shogi and chess as well as Go. Differences between AZ and AGZ include:^[8]

• AZ has hard-coded rules for setting searchhyperparameters.
• The neural network is now updated continually.
• Chess (unlike Go) can end in a tie; therefore AZ can take into account the possibility of a tie game.

Anopen source program,Leela Zero, based on the ideas from the AlphaGo papers is available. It uses aGPU instead of theTPUs recent versions of AlphaGo rely on.

• "AlphaGo Zero: Starting from scratch". Archived fromthe original on 3 January 2020.
• Singh, S.; Okun, A.; Jackson, A. (2017)."AOP".Nature.550 (7676):336–337.Bibcode:2017Natur.550..336S.doi:10.1038/550336a.PMID 29052631.S2CID 4447445.
• Silver, David; Schrittwieser, Julian; Simonyan, Karen; Antonoglou, Ioannis; Huang, Aja; Guez, Arthur; Hubert, Thomas; Baker, Lucas; Lai, Matthew; Bolton, Adrian; Chen, Yutian; Lillicrap, Timothy; Hui, Fan; Sifre, Laurent; Van Den Driessche, George; Graepel, Thore; Hassabis, Demis (2017)."Mastering the game of Go without human knowledge"(PDF).Nature.550 (7676):354–359.Bibcode:2017Natur.550..354S.doi:10.1038/nature24270.PMID 29052630.S2CID 205261034.
• AlphaGo Zero Games

• 2017 software
• Applied machine learning
• AlphaGo
• 2017 in go

• CS1 maint: numeric names: authors list
• CS1 Chinese-language sources (zh)
• Articles with short description
• Short description is different from Wikidata
• Use dmy dates from October 2017