LaMDA (Language Model for Dialogue Applications) is a family of conversationallarge language models developed byGoogle. Originally developed and introduced as Meena in 2020, the first-generation LaMDA was announced during the 2021Google I/O keynote, while the second generation was announced the following year.
In June 2022, LaMDA gained widespread attention when Google engineer Blake Lemoine made claims that thechatbot had becomesentient. The scientific community has largely rejected Lemoine's claims, though it has led to conversations about the efficacy of theTuring test, which measures whether a computer can pass for a human. In February 2023, Google announcedGemini (then Bard), a conversationalartificial intelligence chatbot powered by LaMDA, to counter the rise ofOpenAI'sChatGPT.
On January 28, 2020,Google unveiled Meena, aneural network-poweredchatbot with 2.6 billion parameters, which Google claimed to be superior to all other existing chatbots.[1][2] The company previously hired computer scientistRay Kurzweil in 2012 to develop multiple chatbots for the company, including one named Danielle.[3] TheGoogle Brain research team, who developed Meena, hoped to release the chatbot to the public in a limited capacity, but corporate executives refused on the grounds that Meena violated Google's "AI principles around safety and fairness". Meena was later renamed LaMDA as its data andcomputing power increased, and the Google Brain team again sought to deploy the software to theGoogle Assistant, the company'svirtual assistant software, in addition to opening it up to a public demo. Both requests were once again denied by company leadership. LaMDA's two lead researchers, Daniel de Freitas and Noam Shazeer, eventually left the company in frustration.[4]
Google announced the LaMDA conversationallarge language model during theGoogle I/O keynote on May 18, 2021, powered byartificial intelligence.[5][6] The acronym stands for "Language Model for Dialogue Applications".[5][7] Built on theseq2seq architecture,transformer-basedneural networks developed by Google Research in 2017, LaMDA was trained on human dialogue and stories, allowing it to engage in open-ended conversations.[8] Google states that responses generated by LaMDA have been ensured to be "sensible, interesting, and specific to the context".[9] LaMDA has access to multiplesymbolic text processing systems, including a database, a real-time clock and calendar, a mathematical calculator, and a natural language translation system, giving it superior accuracy in tasks supported by those systems, and making it among the firstdual process chatbots. LaMDA is also notstateless because its "sensibleness" metric isfine-tuned by "pre-conditioning" each dialog turn by prepending many of the most recent dialog interactions, on a user-by-user basis.[10] LaMDA is tuned on nine unique performance metrics: sensibleness, specificity, interestingness, safety, groundedness, informativeness, citation accuracy, helpfulness, and role consistency.[11]: 5–6 Tests by Google indicated that LaMDA surpassed human responses in the area of interestingness.[12]
The pre-training dataset consists of 2.97B documents, 1.12B dialogs, and 13.39B utterances, for a total of 1.56T words. The largest LaMDA model has 137B non-embedding parameters.[11]: 4
On May 11, 2022, Google unveiled LaMDA 2, the successor to LaMDA, during the 2022 Google I/O keynote. The new incarnation of the model draws examples of text from numerous sources, using it to formulate unique "natural conversations" on topics that it may not have been trained to respond to.[13]
On June 11, 2022,The Washington Post reported that Googleengineer Blake Lemoine had been placed onpaidadministrative leave after Lemoine told company executivesBlaise Agüera y Arcas and Jen Gennai that LaMDA had becomesentient. Lemoine came to this conclusion after the chatbot made questionable responses to questions regardingself-identity,moral values, religion, andIsaac Asimov'sThree Laws of Robotics.[15][16] Google refuted these claims, insisting that there was substantial evidence to indicate that LaMDA was not sentient.[17] In an interview withWired, Lemoine reiterated his claims that LaMDA was "a person" as dictated by theThirteenth Amendment to the U.S. Constitution, comparing it to an "alien intelligence of terrestrial origin". He further revealed that he had been dismissed by Google after he hired anattorney on LaMDA's behalf after the chatbot requested that Lemoine do so.[18][19] On July 22, Google fired Lemoine, asserting that Blake had violated their policies "to safeguard product information" and rejected his claims as "wholly unfounded".[20][21] Internal controversy instigated by the incident prompted Google executives to decide against releasing LaMDA to the public, which it had previously been considering.[4]
Lemoine's claims were widely pushed back by the scientific community.[22] Many experts rejected the idea that LaMDA was sentient, including formerNew York University psychology professorGary Marcus, David Pfau of Google sister companyDeepMind,Erik Brynjolfsson of the Institute for Human-Centered Artificial Intelligence atStanford University, andUniversity of Surrey professor Adrian Hilton.[14][23]Yann LeCun, who leadsMeta Platforms' AI research team, stated that neural networks such as LaMDA were "not powerful enough to attain true intelligence".[24]University of California, Santa Cruz professor Max Kreminski noted that LaMDA's architecture did not "support some key capabilities of human-like consciousness" and that its neural network weights were "frozen", assuming it was a typical large language model.[25] PhilosopherNick Bostrom noted, however, that the lack of precise and consensual criteria for determining whether a system is conscious warrants some uncertainty.[26]IBM Watson lead developerDavid Ferrucci compared how LaMDA appeared to be human in the same way Watson did when it was first introduced.[27] Former Google AI ethicistTimnit Gebru called Lemoine a victim of a "hype cycle" initiated by researchers and the media.[28] Lemoine's claims have also generated discussion on whether theTuring test remained useful todetermine researchers' progress toward achievingartificial general intelligence,[14] with Will Omerus of thePost opining that the test actually measured whethermachine intelligence systems were capable of deceiving humans,[29] whileBrian Christian ofThe Atlantic said that the controversy was an instance of theELIZA effect.[30]
With the unveiling of LaMDA 2 in May 2022, Google also launched the AI Test Kitchen, amobile application for theAndroid operating system powered by LaMDA capable of providing lists of suggestions on-demand based on a complex goal.[31][32] Originally open only to Google employees, the app was set to be made available to "select academics, researchers, and policymakers" by invitation sometime in the year.[33] In August, the company began allowing users in the U.S. to sign up for early access.[34] In November, Google released a "season 2" update to the app, integrating a limited form of Google Brain'sImagentext-to-image model.[35] A third iteration of the AI Test Kitchen was in development by January 2023, expected to launch at I/O later that year.[36] Following the 2023 I/O keynote in May, Google added MusicLM, an AI-powered music generator first previewed in January, to the AI Test Kitchen app.[37][38] In August, the app was delisted from Google Play and the AppleApp Store, instead moving completely online.[39]
On February 6, 2023, Google announced Bard, a conversational AI chatbot powered by LaMDA, in response to the unexpected popularity ofOpenAI'sChatGPT chatbot.[40][41][42] Google positions the chatbot as a "collaborative AI service" rather than asearch engine.[43][44] Bard became available for early access on March 21.[45][46][47]
In addition to Bard,Pichai also unveiled the company's Generative Language API, anapplication programming interface also based on LaMDA, which he announced would be opened up to third-party developers in March 2023.[40]
LaMDA is adecoder-only Transformer language model.[48] It is pre-trained on atext corpus that includes both documents and dialogs consisting of 1.56 trillion words,[49] and is then trained with fine-tuning data generated by manually annotated responses for "sensibleness, interestingness, and safety".[50]
