This is an accepted version of this page
Speech synthesis is the artificial production of humanspeech. A computer system used for this purpose is called aspeech synthesizer, and can be implemented insoftware orhardware products. Atext-to-speech (TTS) system converts normal language text into speech; other systems rendersymbolic linguistic representations likephonetic transcriptions into speech.[1] The reverse process isspeech recognition.
Synthesized speech can be created byconcatenating pieces of recorded speech that are stored in adatabase. Systems differ in the size of the stored speech units; a system that storesphones ordiphones provides the largest output range, but may lack clarity.[citation needed] For specific usage domains, the storage of entire words or sentences allows for high-quality output. Alternatively, a synthesizer can incorporate a model of thevocal tract and other human voice characteristics to create a completely "synthetic" voice output.[2]
The quality of a speech synthesizer is judged by its similarity to the human voice and by its ability to be understood clearly. An intelligible text-to-speech program allows people withvisual impairments orreading disabilities to listen to written words on a home computer. The earliest computeroperating system to have included a speech synthesizer wasUnix in 1974, through the Unixspeak utility.[3] In 2000, Microsoft Sam was the defaulttext-to-speech voice synthesizer used by thenarrator accessibility feature, which shipped with all Windows 2000 operating systems, and subsequent Windows XP systems.
A text-to-speech system (or "engine") is composed of two parts:[4] afront-end and aback-end. The front-end has two major tasks. First, it converts raw text containing symbols like numbers and abbreviations into the equivalent of written-out words. This process is often calledtext normalization,pre-processing, ortokenization. The front-end then assignsphonetic transcriptions to each word, and divides and marks the text intoprosodic units, likephrases,clauses, andsentences. The process of assigning phonetic transcriptions to words is calledtext-to-phoneme orgrapheme-to-phoneme conversion. Phonetic transcriptions andprosody information together make up the symbolic linguistic representation that is output by the front-end. The back-end—often referred to as thesynthesizer—then converts the symbolic linguistic representation into sound. In certain systems, this part includes the computation of thetarget prosody (pitch contour, phoneme durations),[5] which is then imposed on the output speech.
Long before the invention ofelectronicsignal processing, some people tried to build machines to emulate human speech.[6][better source needed] There were also legends of the existence of "Brazen Heads", such as those involving PopeSilvester II (d. 1003 AD),Albertus Magnus (1198–1280), andRoger Bacon (1214–1294).[7]
In 1779, theGerman-Danish scientistChristian Gottlieb Kratzenstein won the first prize in a competition announced by the RussianImperial Academy of Sciences and Arts for models he built of the humanvocal tract that could produce the five longvowel sounds (inInternational Phonetic Alphabet notation:[aː],[eː],[iː],[oː] and[uː]).[8] There followed thebellows-operated "acoustic-mechanical speech machine" ofWolfgang von Kempelen ofPressburg, Hungary, described in a 1791 paper.[9] This machine added models of the tongue and lips, enabling it to produce consonants as well as vowels. In 1837,Charles Wheatstone produced a "speaking machine" based on von Kempelen's design, and in 1846, Joseph Faber exhibited the "Euphonia". In 1923, Paget resurrected Wheatstone's design.[10]
In the 1930s,Bell Labs developed thevocoder, which automatically analyzed speech into its fundamental tones and resonances. From his work on the vocoder,Homer Dudley developed a keyboard-operated voice-synthesizer calledThe Voder (Voice Demonstrator), which he exhibited at the1939 New York World's Fair.
Dr. Franklin S. Cooper and his colleagues atHaskins Laboratories built thePattern playback in the late 1940s and completed it in 1950. There were several different versions of this hardware device; only one currently survives. The machine converts pictures of the acoustic patterns of speech in the form of a spectrogram back into sound. Using this device,Alvin Liberman and colleagues discovered acoustic cues for the perception ofphonetic segments (consonants and vowels).
.jpg)
The first computer-based speech-synthesis systems originated in the late 1950s. Noriko Umedaet al. developed the first general English text-to-speech system in 1968, at theElectrotechnical Laboratory in Japan.[11] In 1961, physicistJohn Larry Kelly, Jr and his colleagueLouis Gerstman[12] used anIBM 704 computer to synthesize speech, an event among the most prominent in the history ofBell Labs.[citation needed] Kelly's voice recorder synthesizer (vocoder) recreated the song "Daisy Bell", with musical accompaniment fromMax Mathews. Coincidentally,Arthur C. Clarke was visiting his friend and colleague John Pierce at the Bell Labs Murray Hill facility. Clarke was so impressed by the demonstration that he used it in the climactic scene of his screenplay for his novel2001: A Space Odyssey,[13] where theHAL 9000 computer sings the same song as astronautDave Bowman puts it to sleep.[14] Despite the success of purely electronic speech synthesis, research into mechanical speech-synthesizers continues.[15][independent source needed]
Linear predictive coding (LPC), a form ofspeech coding, began development with the work ofFumitada Itakura ofNagoya University and Shuzo Saito ofNippon Telegraph and Telephone (NTT) in 1966. Further developments in LPC technology were made byBishnu S. Atal andManfred R. Schroeder atBell Labs during the 1970s.[16] LPC was later the basis for early speech synthesizer chips, such as theTexas Instruments LPC Speech Chips used in theSpeak & Spell toys from 1978.
In 1975, Fumitada Itakura developed theline spectral pairs (LSP) method for high-compression speech coding, while at NTT.[17][18][19] From 1975 to 1981, Itakura studied problems in speech analysis and synthesis based on the LSP method.[19] In 1980, his team developed an LSP-based speech synthesizer chip. LSP is an important technology for speech synthesis and coding, and in the 1990s was adopted by almost all international speech coding standards as an essential component, contributing to the enhancement of digital speech communication over mobile channels and the internet.[18]
In 1975,MUSA was released, and was one of the first Speech Synthesis systems. It consisted of a stand-alone computer hardware and a specialized software that enabled it to read Italian. A second version, released in 1978, was also able to sing Italian in an "a cappella" style.[20]
Dominant systems in the 1980s and 1990s were theDECtalk system, based largely on the work ofDennis Klatt at MIT, and the Bell Labs system;[21] the latter was one of the first multilingual language-independent systems, making extensive use ofnatural language processing methods.
.png)
Handheld electronics featuring speech synthesis began emerging in the 1970s. One of the first was theTelesensory Systems Inc. (TSI)Speech+ portable calculator for the blind in 1976.[22][23] Other devices had primarily educational purposes, such as theSpeak & Spell toy produced byTexas Instruments in 1978.[24] Fidelity released a speaking version of its electronic chess computer in 1979.[25] The firstvideo game to feature speech synthesis was the 1980shoot 'em uparcade game,Stratovox (known in Japan asSpeak & Rescue), fromSun Electronics.[26][27] The firstpersonal computer game with speech synthesis wasManbiki Shoujo (Shoplifting Girl), released in 1980 for thePET 2001, for which the game's developer, Hiroshi Suzuki, developed a "zero cross" programming technique to produce a synthesized speech waveform.[28] Another early example, the arcade version ofBerzerk, also dates from 1980. TheMilton Bradley Company produced the first multi-playerelectronic game using voice synthesis,Milton, in the same year.
In 1976, Computalker Consultants released their CT-1 Speech Synthesizer. Designed by D. Lloyd Rice and Jim Cooper, it was an analog synthesizer built to work with microcomputers using the S-100 bus standard.[29]
Synthesized voices typically sounded male until 1990, whenAnn Syrdal, atAT&T Bell Laboratories, created a female voice.[30]
Kurzweil predicted in 2005 that as thecost-performance ratio caused speech synthesizers to become cheaper and more accessible, more people would benefit from the use of text-to-speech programs.[31]
In September 2016,DeepMind releasedWaveNet, which demonstrated thatdeep learning models are capable of modeling raw waveforms and generating speech from acoustic features likespectrograms ormel-spectrograms, starting the field ofdeep learning speech synthesis. Although WaveNet was initially considered to be computationally expensive and slow to be used in consumer products at the time, a year after its release, DeepMind unveiled a modified version of WaveNet known as "Parallel WaveNet," a production model 1,000 times faster than the original.[32] This was followed byGoogle AI's Tacotron 2 in 2018, which demonstrated that neural networks could produce highly natural speech synthesis but required substantial training data—typically tens of hours of audio—to achieve acceptable quality. Tacotron 2 employed anautoencoder architecture withattention mechanisms to convert input text into mel-spectrograms, which were then converted to waveforms using a separate neuralvocoder. When trained on smaller datasets, such as 2 hours of speech, the output quality degraded while still being able to maintain intelligible speech, and with just 24 minutes of training data, Tacotron 2 failed to produce intelligible speech.[33]
In 2019,Microsoft Research introduced FastSpeech, which addressed speed limitations inautoregressive models like Tacotron 2.[34] The same year saw the release of HiFi-GAN, agenerative adversarial network (GAN)-based vocoder that improved the efficiency of waveform generation while producing high-fidelity speech.[35] In 2020, the release of Glow-TTS introduced aflow-based approach that allowed for both fast inference and voice style transfer capabilities.[36]
In March 2020, the free text-to-speech website15.ai was launched. 15.ai gained widespread international attention in early 2021 for its ability to synthesize emotionally expressive speech of fictional characters from popular media with minimal amount of data.[37][38][39] The creator of 15.ai stated that 15 seconds of training data is sufficient to perfectly clone a person's voice (hence its name, "15.ai"), a significant reduction from the previously known data requirement of tens of hours.[40] 15.ai is credited as the first platform to popularize AI voice cloning inmemes andcontent creation.[41][42][40] In January 2022, the first instance of speech synthesisNFT fraud occurred when acryptocurrency company calledVoiceverse generated voice lines using 15.ai, pitched them up to sound unrecognizable, promoted them as the byproduct of their own technology, and sold them asNFTs without permission.[43][44][45][46]
In January 2023,ElevenLabs launched its browser-based text-to-speech platform, which employs advanced algorithms to analyze contextual aspects of text and detect emotions such as anger, sadness, happiness, or alarm.[47][48][49] The platform is able to adjust intonation and pacing based on linguistic context to produce lifelike speech with human-like inflection, and offers features including multilingual speech generation and long-form content creation.[50][51]
In March 2024,OpenAI corroborated the 15 second benchmark to clone a human's voice.[52] However, they deemed their Voice Engine tool "too risky" for general release, stating that they would only release a preview and not release the technology for public use.[53]
The most important qualities of a speech synthesis system arenaturalness andintelligibility.[54] Naturalness describes how closely the output sounds like human speech, while intelligibility is the ease with which the output is understood. The ideal speech synthesizer is both natural and intelligible. Speech synthesis systems usually try to maximize both characteristics.
The two primary technologies generating synthetic speech waveforms areconcatenative synthesis andformant synthesis. Each technology has strengths and weaknesses, and the intended uses of a synthesis system will typically determine which approach is used.
Concatenative synthesis is based on the concatenation (stringing together) of segments of recorded speech. Generally, concatenative synthesis produces the most natural-sounding synthesized speech. However, differences between natural variations in speech and the nature of the automated techniques for segmenting the waveforms sometimes result in audible glitches in the output. There are three main sub-types of concatenative synthesis.
Unit selection synthesis uses large databases of recorded speech. During database creation, each recorded utterance is segmented into some or all of the following: individualphones,diphones, half-phones,syllables,morphemes,words,phrases, andsentences. Typically, the division into segments is done using a specially modifiedspeech recognizer set to a "forced alignment" mode with some manual correction afterward, using visual representations such as thewaveform andspectrogram.[55] Anindex of the units in the speech database is then created based on the segmentation and acoustic parameters like thefundamental frequency (pitch), duration, position in the syllable, and neighboring phones. Atrun time, the desired target utterance is created by determining the best chain of candidate units from the database (unit selection). This process is typically achieved using a specially weighteddecision tree.
Unit selection provides the greatest naturalness, because it applies only a small amount ofdigital signal processing (DSP) to the recorded speech. DSP often makes recorded speech sound less natural, although some systems use a small amount of signal processing at the point of concatenation to smooth the waveform. The output from the best unit-selection systems is often indistinguishable from real human voices, especially in contexts for which the TTS system has been tuned. However, maximum naturalness typically require unit-selection speech databases to be very large, in some systems ranging into thegigabytes of recorded data, representing dozens of hours of speech.[56] Also, unit selection algorithms have been known to select segments from a place that results in less than ideal synthesis (e.g. minor words become unclear) even when a better choice exists in the database.[57] Recently, researchers have proposed various automated methods to detect unnatural segments in unit-selection speech synthesis systems.[58]
Diphone synthesis uses a minimal speech database containing all thediphones (sound-to-sound transitions) occurring in a language. The number of diphones depends on thephonotactics of the language: for example, Spanish has about 800 diphones, and German about 2500. In diphone synthesis, only one example of each diphone is contained in the speech database. At runtime, the targetprosody of a sentence is superimposed on these minimal units by means ofdigital signal processing techniques such aslinear predictive coding,PSOLA[59] orMBROLA.[60] or more recent techniques such as pitch modification in the source domain usingdiscrete cosine transform.[61] Diphone synthesis suffers from the sonic glitches of concatenative synthesis and the robotic-sounding nature of formant synthesis, and has few of the advantages of either approach other than small size. As such, its use in commercial applications is declining,[citation needed] although it continues to be used in research because there are a number of freely available software implementations. An early example of Diphone synthesis is a teaching robot,Leachim, that was invented byMichael J. Freeman.[62] Leachim contained information regarding class curricular and certain biographical information about the students whom it was programmed to teach.[63] It was tested in a fourth grade classroom inthe Bronx, New York.[64][65]
Domain-specific synthesis concatenates prerecorded words and phrases to create complete utterances. It is used in applications where the variety of texts the system will output is limited to a particular domain, like transit schedule announcements or weather reports.[66] The technology is very simple to implement, and has been in commercial use for a long time, in devices like talking clocks and calculators. The level of naturalness of these systems can be very high because the variety of sentence types is limited, and they closely match the prosody and intonation of the original recordings.[citation needed]
Because these systems are limited by the words and phrases in their databases, they are not general-purpose and can only synthesize the combinations of words and phrases with which they have been preprogrammed. The blending of words within naturally spoken language however can still cause problems unless the many variations are taken into account. For example, innon-rhotic dialects of English the"r" in words like"clear"/ˈklɪə/ is usually only pronounced when the following word has a vowel as its first letter (e.g."clear out" is realized as/ˌklɪəɹˈʌʊt/). Likewise inFrench, many final consonants become no longer silent if followed by a word that begins with a vowel, an effect calledliaison. Thisalternation cannot be reproduced by a simple word-concatenation system, which would require additional complexity to becontext-sensitive.
Formant synthesis does not use human speech samples at runtime. Instead, the synthesized speech output is created usingadditive synthesis and an acoustic model (physical modelling synthesis).[67] Parameters such asfundamental frequency,voicing, andnoise levels are varied over time to create awaveform of artificial speech. This method is sometimes calledrules-based synthesis; however, many concatenative systems also have rules-based components.Many systems based on formant synthesis technology generate artificial, robotic-sounding speech that would never be mistaken for human speech. However, maximum naturalness is not always the goal of a speech synthesis system, and formant synthesis systems have advantages over concatenative systems. Formant-synthesized speech can be reliably intelligible, even at very high speeds, avoiding the acoustic glitches that commonly plague concatenative systems. High-speed synthesized speech is used by the visually impaired to quickly navigate computers using ascreen reader. Formant synthesizers are usually smaller programs than concatenative systems because they do not have a database of speech samples. They can therefore be used inembedded systems, wherememory andmicroprocessor power are especially limited. Because formant-based systems have complete control of all aspects of the output speech, a wide variety of prosodies andintonations can be output, conveying not just questions and statements, but a variety of emotions and tones of voice.
Examples of non-real-time but highly accurate intonation control in formant synthesis include the work done in the late 1970s for theTexas Instruments toySpeak & Spell, and in the early 1980sSegaarcade machines[68] and in manyAtari, Inc. arcade games[69] using theTMS5220 LPC Chips. Creating proper intonation for these projects was painstaking, and the results have yet to be matched by real-time text-to-speech interfaces.[70][when?]
For tonal languages, such as Chinese or Taiwanese language, there are different levels oftone sandhi required and sometimes the output of speech synthesizer may result in the mistakes of tone sandhi.[71]
Articulatory synthesis consists of computational techniques for synthesizing speech based on models of the humanvocal tract and the articulation processes occurring there. The first articulatory synthesizer regularly used for laboratory experiments was developed atHaskins Laboratories in the mid-1970s byPhilip Rubin, Tom Baer, and Paul Mermelstein. This synthesizer, known as ASY, was based on vocal tract models developed atBell Laboratories in the 1960s and 1970s by Paul Mermelstein, Cecil Coker, and colleagues.
Until recently, articulatory synthesis models have not been incorporated into commercial speech synthesis systems. A notable exception is theNeXT-based system originally developed and marketed by Trillium Sound Research, a spin-off company of theUniversity of Calgary, where much of the original research was conducted. Following the demise of the various incarnations of NeXT (started bySteve Jobs in the late 1980s and merged with Apple Computer in 1997), the Trillium software was published under the GNU General Public License, with work continuing asgnuspeech. The system, first marketed in 1994, provides full articulatory-based text-to-speech conversion using a waveguide or transmission-line analog of the human oral and nasal tracts controlled by Carré's "distinctive region model".
More recent synthesizers, developed by Jorge C. Lucero and colleagues, incorporate models of vocal fold biomechanics, glottal aerodynamics and acoustic wave propagation in the bronchi, trachea, nasal and oral cavities, and thus constitute full systems of physics-based speech simulation.[72][73]
HMM-based synthesis is a synthesis method based onhidden Markov models, also called Statistical Parametric Synthesis. In this system, thefrequency spectrum (vocal tract),fundamental frequency (voice source), and duration (prosody) of speech are modeled simultaneously by HMMs. Speechwaveforms are generated from HMMs themselves based on themaximum likelihood criterion.[74]
Sinewave synthesis is a technique for synthesizing speech by replacing theformants (main bands of energy) with pure tone whistles.[75]
Deep learning speech synthesis usesdeep neural networks (DNN) to produce artificial speech from text (text-to-speech) or spectrum (vocoder).The deep neural networks are trained using a large amount of recorded speech and, in the case of a text-to-speech system, the associated labels and/or input text.
| Part ofa series on |
| Artificial intelligence (AI) |
|---|
_artificial_intelligence_icon.png){kind=icon} |
Glossary |
In 2023,VICE reporterJoseph Cox published findings that he had recorded five minutes of himself talking and then used a tool developed by ElevenLabs to create voice deepfakes that defeated a bank'svoice-authentication system.[83]
The process of normalizing text is rarely straightforward. Texts are full ofheteronyms,numbers, andabbreviations that all require expansion into a phonetic representation. There are many spellings in English which are pronounced differently based on context. For example, "My latest project is to learn how to better project my voice" contains two pronunciations of "project".
Most text-to-speech (TTS) systems do not generatesemantic representations of their input texts, as processes for doing so are unreliable, poorly understood, and computationally ineffective. As a result, variousheuristic techniques are used to guess the proper way to disambiguatehomographs, like examining neighboring words and using statistics about frequency of occurrence.
Recently TTS systems have begun to use HMMs (discussedabove) to generate "parts of speech" to aid in disambiguating homographs. This technique is quite successful for many cases such as whether "read" should be pronounced as "red" implying past tense, or as "reed" implying present tense. Typical error rates when using HMMs in this fashion are usually below five percent. These techniques also work well for most European languages, although access to required trainingcorpora is frequently difficult in these languages.
Deciding how to convert numbers is another problem that TTS systems have to address. It is a simple programming challenge to convert a number into words (at least in English), like "1325" becoming "one thousand three hundred twenty-five". However, numbers occur in many different contexts; "1325" may also be read as "one three two five", "thirteen twenty-five" or "thirteen hundred and twenty five". A TTS system can often infer how to expand a number based on surrounding words, numbers, and punctuation, and sometimes the system provides a way to specify the context if it is ambiguous.[84] Roman numerals can also be read differently depending on context. For example, "Henry VIII" reads as "Henry the Eighth", while "Chapter VIII" reads as "Chapter Eight".
Similarly, abbreviations can be ambiguous. For example, the abbreviation "in" for "inches" must be differentiated from the word "in", and the address "12 St John St." uses the same abbreviation for both "Saint" and "Street". TTS systems with intelligent front ends can make educated guesses about ambiguous abbreviations, while others provide the same result in all cases, resulting in nonsensical (and sometimes comical) outputs, such as "Ulysses S. Grant" being rendered as "Ulysses South Grant".
 | This sectiondoes notcite anysources. Please helpimprove this section byadding citations to reliable sources. Unsourced material may be challenged andremoved.(April 2023) (Learn how and when to remove this message) |
Speech synthesis systems use two basic approaches to determine the pronunciation of a word based on itsspelling, a process which is often called text-to-phoneme orgrapheme-to-phoneme conversion (phoneme is the term used bylinguists to describe distinctive sounds in alanguage). The simplest approach to text-to-phoneme conversion is the dictionary-based approach, where a large dictionary containing all the words of a language and their correctpronunciations is stored by the program. Determining the correct pronunciation of each word is a matter of looking up each word in the dictionary and replacing the spelling with the pronunciation specified in the dictionary. The other approach is rule-based, in which pronunciation rules are applied to words to determine their pronunciations based on their spellings. This is similar to the "sounding out", orsynthetic phonics, approach to learning reading.
Each approach has advantages and drawbacks. The dictionary-based approach is quick and accurate, but completely fails if it is given a word which is not in its dictionary. As dictionary size grows, so too does the memory space requirements of the synthesis system. On the other hand, the rule-based approach works on any input, but the complexity of the rules grows substantially as the system takes into account irregular spellings or pronunciations. (Consider that the word "of" is very common in English, yet is the only word in which the letter "f" is pronounced[v].) As a result, nearly all speech synthesis systems use a combination of these approaches.
Languages with aphonemic orthography have a very regular writing system, and the prediction of the pronunciation of words based on their spellings is quite successful. Speech synthesis systems for such languages often use the rule-based method extensively, resorting to dictionaries only for those few words, like foreign names and loanwords, whose pronunciations are not obvious from their spellings. On the other hand, speech synthesis systems for languages like English, which have extremely irregular spelling systems, are more likely to rely on dictionaries, and to use rule-based methods only for unusual words, or words that are not in their dictionaries.
The consistent evaluation of speech synthesis systems may be difficult because of a lack of universally agreed objective evaluation criteria. Different organizations often use different speech data. The quality of speech synthesis systems also depends on the quality of the production technique (which may involve analogue or digital recording) and on the facilities used to replay the speech. Evaluating speech synthesis systems has therefore often been compromised by differences between production techniques and replay facilities.
Since 2005, however, some researchers have started to evaluate speech synthesis systems using a common speech dataset.[85]
A study in the journalSpeech Communication by Amy Drahota and colleagues at theUniversity of Portsmouth,UK, reported that listeners to voice recordings could determine, at better than chance levels, whether or not the speaker was smiling.[86][87][88] It was suggested that identification of the vocal features that signal emotional content may be used to help make synthesized speech sound more natural. One of the related issues is modification of thepitch contour of the sentence, depending upon whether it is an affirmative, interrogative or exclamatory sentence. One of the techniques for pitch modification[61] usesdiscrete cosine transform in the source domain (linear prediction residual). Such pitch synchronous pitch modification techniques need a priori pitch marking of the synthesis speech database using techniques such as epoch extraction using dynamicplosion index applied on the integrated linear prediction residual of thevoiced regions of speech.[89] In general, prosody remains a challenge for speech synthesizers, and is an active research topic.
Popular systems offering speech synthesis as a built-in capability.
In the early 1980s, TI was known as a pioneer in speech synthesis, and a highly popular plug-in speech synthesizer module was available for the TI-99/4 and 4A. Speech synthesizers were offered free with the purchase of a number of cartridges and were used by many TI-written video games (games offered with speech during this promotion includedAlpiner andParsec). The synthesizer uses a variant of linear predictive coding and has a small in-built vocabulary. The original intent was to release small cartridges that plugged directly into the synthesizer unit, which would increase the device's built-in vocabulary. However, the success of software text-to-speech in the Terminal Emulator II cartridge canceled that plan.
TheMattelIntellivision game console offered theIntellivoice Voice Synthesis module in 1982. It included theSP0256 Narrator speech synthesizer chip on a removable cartridge. The Narrator had 2kB of Read-Only Memory (ROM), and this was utilized to store a database of generic words that could be combined to make phrases in Intellivision games. Since the Orator chip could also accept speech data from external memory, any additional words or phrases needed could be stored inside the cartridge itself. The data consisted of strings of analog-filter coefficients to modify the behavior of the chip's synthetic vocal-tract model, rather than simple digitized samples.
Also released in 1982,Software Automatic Mouth was the first commercial all-software voice synthesis program. It was later used as the basis forMacintalk. The program was available for non-Macintosh Apple computers (including the Apple II, and the Lisa), various Atari models and the Commodore 64. The Apple version preferred additional hardware that contained DACs, although it could instead use the computer's one-bit audio output (with the addition of much distortion) if the card was not present. The Atari made use of the embedded POKEY audio chip. Speech playback on the Atari normally disabled interrupt requests and shut down the ANTIC chip during vocal output. The audible output is extremely distorted speech when the screen is on. The Commodore 64 made use of the 64's embedded SID audio chip.
Arguably, the first speech system integrated into anoperating system was the circa 1983 unreleased Atari1400XL/1450XL computers. These used the Votrax SC01 chip and afinite-state machine to enable World English Spelling text-to-speech synthesis.[91]
TheAtari ST computers were sold with "stspeech.tos" on floppy disk.
The first speech system integrated into anoperating system that shipped in quantity wasApple Computer'sMacInTalk. The software was licensed from third-party developers Joseph Katz and Mark Barton (later, SoftVoice, Inc.) and was featured during the 1984 introduction of the Macintosh computer. This January demo required 512 kilobytes of RAM memory. As a result, it could not run in the 128 kilobytes of RAM the first Mac actually shipped with.[92] So, the demo was accomplished with a prototype 512k Mac, although those in attendance were not told of this and the synthesis demo created considerable excitement for the Macintosh. In the early 1990s Apple expanded its capabilities offering system wide text-to-speech support. With the introduction of faster PowerPC-based computers they included higher quality voice sampling. Apple also introducedspeech recognition into its systems which provided a fluid command set. More recently, Apple has added sample-based voices. Starting as a curiosity, the speech system of AppleMacintosh has evolved into a fully supported program,PlainTalk, for people with vision problems.VoiceOver was for the first time featured in 2005 inMac OS X Tiger (10.4). During 10.4 (Tiger) and first releases of 10.5 (Leopard) there was only one standard voice shipping with Mac OS X. Starting with 10.6 (Snow Leopard), the user can choose out of a wide range list of multiple voices. VoiceOver voices feature the taking of realistic-sounding breaths between sentences, as well as improved clarity at high read rates over PlainTalk. Mac OS X also includessay, acommand-line based application that converts text to audible speech. TheAppleScript Standard Additions includes a say verb that allows a script to use any of the installed voices and to control the pitch, speaking rate and modulation of the spoken text.
Used inAlexa and asSoftware as a Service in AWS[93] (from 2017).
The second operating system to feature advanced speech synthesis capabilities wasAmigaOS, introduced in 1985. The voice synthesis was licensed byCommodore International from SoftVoice, Inc., who also developed the originalMacinTalk text-to-speech system. It featured a complete system of voice emulation for American English, with both male and female voices and "stress" indicator markers, made possible through theAmiga's audiochipset.[94] The synthesis system was divided into a translator library which converted unrestricted English text into a standard set of phonetic codes and a narrator device which implemented a formant model of speech generation.. AmigaOS also featured a high-level "Speak Handler", which allowed command-line users to redirect text output to speech. Speech synthesis was occasionally used in third-party programs, particularly word processors and educational software. The synthesis software remained largely unchanged from the first AmigaOS release and Commodore eventually removed speech synthesis support from AmigaOS 2.1 onward.
Despite the American English phoneme limitation, an unofficial version with multilingual speech synthesis was developed. This made use of an enhanced version of the translator library which could translate a number of languages, given a set of rules for each language.[95]
ModernWindows desktop systems can useSAPI 4 andSAPI 5 components to support speech synthesis andspeech recognition. SAPI 4.0 was available as an optional add-on forWindows 95 andWindows 98.Windows 2000 addedNarrator, a text-to-speech utility for people who have visual impairment. Third-party programs such as JAWS for Windows, Window-Eyes, Non-visual Desktop Access, Supernova and System Access can perform various text-to-speech tasks such as reading text aloud from a specified website, email account, text document, the Windows clipboard, the user's keyboard typing, etc. Not all programs can use speech synthesis directly.[96] Some programs can use plug-ins, extensions or add-ons to read text aloud. Third-party programs are available that can read text from the system clipboard.
Microsoft Speech Server is a server-based package for voice synthesis and recognition. It is designed for network use withweb applications andcall centers.
From 1971 to 1996, Votrax produced a number of commercial speech synthesizer components. A Votrax synthesizer was included in the first generation Kurzweil Reading Machine for the Blind.
Text-to-speech (TTS) refers to the ability of computers to read text aloud. A TTS engine converts written text to a phonemic representation, then converts the phonemic representation to waveforms that can be output as sound. TTS engines with different languages, dialects and specialized vocabularies are available through third-party publishers.[97]
Version 1.6 ofAndroid added support for speech synthesis (TTS).[98]
Currently, there are a number ofapplications,plugins and gadgets that can read messages directly from ane-mail client and web pages from aweb browser orGoogle Toolbar. Some specialized software can narrateRSS-feeds. On one hand, online RSS-narrators simplify information delivery by allowing users to listen to their favourite news sources and to convert them topodcasts. On the other hand, on-line RSS-readers are available on almost any personal computer connected to the Internet. Users can download generated audio files to portable devices, e.g. with a help ofpodcast receiver, and listen to them while walking, jogging or commuting to work.
A growing field in Internet based TTS is web-basedassistive technology, e.g. 'Browsealoud' from a UK company andReadspeaker. It can deliver TTS functionality to anyone (for reasons of accessibility, convenience, entertainment or information) with access to a web browser. The non-profit projectPediaphon was created in 2006 to provide a similar web-based TTS interface to the Wikipedia.[99]
Other work is being done in the context of theW3C through the W3C Audio Incubator Group with the involvement of The BBC and Google Inc.
Someopen-source software systems are available, such as:
At the 2018Conference on Neural Information Processing Systems (NeurIPS) researchers fromGoogle presented the work 'Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis', whichtransfers learning fromspeaker verification to achieve text-to-speech synthesis, that can be made to sound almost like anybody from a speech sample of only 5 seconds.[102]
Also researchers fromBaidu Research presented avoice cloning system with similar aims at the 2018 NeurIPS conference,[103] though the result is rather unconvincing.
By 2019 the digital sound-alikes found their way to the hands of criminals asSymantec researchers know of 3 cases where digital sound-alikes technology has been used for crime.[104][105]
This increases the stress on the disinformation situation coupled with the facts that
A number ofmarkup languages have been established for the rendition of text as speech in anXML-compliant format. The most recent isSpeech Synthesis Markup Language (SSML), which became aW3C recommendation in 2004. Older speech synthesis markup languages include Java Speech Markup Language (JSML) andSABLE. Although each of these was proposed as a standard, none of them have been widely adopted.[citation needed]
Speech synthesis markup languages are distinguished from dialogue markup languages.VoiceXML, for example, includes tags related to speech recognition, dialogue management and touchtone dialing, in addition to text-to-speech markup.[citation needed]
Speech synthesis has long been a vital assistive technology tool and its application in this area is significant and widespread. It allows environmental barriers to be removed for people with a wide range of disabilities. The longest application has been in the use ofscreen readers for people with visual impairment, but text-to-speech systems are now commonly used by people withdyslexia and otherreading disabilities as well as by pre-literate children.[108] They are also frequently employed to aid those with severespeech impairment usually through a dedicatedvoice output communication aid.[109] Work to personalize a synthetic voice to better match a person's personality or historical voice is becoming available.[110] A noted application, of speech synthesis, was theKurzweil Reading Machine for the Blind which incorporated text-to-phonetics software based on work fromHaskins Laboratories and a black-box synthesizer built byVotrax.[111]
Speech synthesis techniques are also used in entertainment productions such as games and animations. In 2007, Animo Limited announced the development of a software application package based on its speech synthesis software FineSpeech, explicitly geared towards customers in the entertainment industries, able to generate narration and lines of dialogue according to user specifications.[112] The application reached maturity in 2008, when NECBiglobe announced a web service that allows users to create phrases from the voices of characters from the Japaneseanime seriesCode Geass: Lelouch of the Rebellion R2.[113] 15.ai has been frequently used forcontent creation in variousfandoms, including theMy Little Pony: Friendship Is Magic fandom, theTeam Fortress 2 fandom, thePortal fandom, and theSpongeBob SquarePants fandom.[114][115][116]
Text-to-speech for disability and impaired communication aids have become widely available. Text-to-speech is also finding new applications; for example, speech synthesis combined withspeech recognition allows for interaction with mobile devices vianatural language processing interfaces. Some users have also created AIvirtual assistants using 15.ai and external voice control software.[37][117]
Text-to-speech is also used in second language acquisition. Voki, for instance, is an educational tool created by Oddcast that allows users to create their own talking avatar, using different accents. They can be emailed, embedded on websites or shared on social media.
Content creators have used voice cloning tools to recreate their voices for podcasts,[118][119] narration,[49] and comedy shows.[120][121][122] Publishers and authors have also used such software to narrate audiobooks and newsletters.[123][124] Another area of application is AI video creation with talking heads. Webapps and video editors like Elai.io orSynthesia allow users to create video content involving AI avatars, who are made to speak using text-to-speech technology.[125][126]
Speech synthesis is a valuable computational aid for the analysis and assessment of speech disorders. Avoice quality synthesizer, developed by Jorge C. Lucero et al. at theUniversity of Brasília, simulates the physics ofphonation and includes models of vocal frequency jitter and tremor, airflow noise and laryngeal asymmetries.[72] The synthesizer has been used to mimic thetimbre ofdysphonic speakers with controlled levels of roughness, breathiness and strain.[73]
