text-to-speech
In plain wordsTechnology that reads written text aloud in a computer-generated voice, used in screen readers, voice assistants, audiobooks and navigation apps.
The automatic production of spoken audio from written text. Classic systems split the job in two: a front end decides what is to be said and how, reading '$5' as 'five dollars' or 'Dr.' as 'doctor' or 'drive', looking up or predicting each word's pronunciation and planning rhythm, stress and pitch, while a back end turns that plan into a waveform. Back ends evolved from rule-based formant synthesis (1970s–80s), through concatenation of recorded speech units chosen from large databases (unit selection, 1990s) and statistical parametric synthesis with hidden Markov modelshidden Markov model, n.: A model of a sequence, such as speech or DNA, in which hidden states — say, the words being spoken — change step by step at random, and each state gives off signals that can be observed, such as sounds. (2000s), to neural synthesis: WaveNetWaveNet, prop. n.: A DeepMind model that generates speech and other sounds one tiny audio sample at a time, producing voices far more natural than earlier computer speech. (2016) generated raw audio; Tacotron (2017) predicted mel spectrogramsmel spectrogram, n.: A version of a sound picture whose pitch axis is squeezed to match human hearing, giving more detail to low sounds than to high ones; the usual way AI models take in audio. from characters for a neural vocodervocoder, n.: A device or program that turns a compact description of a voice back into actual sound; in modern speech AI, the final stage that produces the audio you hear.; non-autoregressive models such as FastSpeech (2019) and end-to-end VITS (2021) improved speed; and from 2023 'codec language models' such as VALL-E generated neural audio codecneural audio codec, n.: A neural network that squeezes sound into a very compact stream of numbers and rebuilds it, used both to send audio cheaply and to let AI models 'write' speech and music piece by piece. tokens, imitating a new voice from a few seconds of audio. Modern systems produce speech that listeners often cannot tell from human recordings in short samples, with controllable emotion and style and low latency for streaming. Quality is judged by mean opinion scoremean opinion score, n.: A quality score from 1 (bad) to 5 (excellent) obtained by asking many listeners to rate audio, such as a synthetic voice or a phone call, and averaging their ratings. and intelligibility tests. The same advances enable voice cloningvoice cloning, n.: Using AI to copy a specific person's voice from recordings, so a computer can say new things that sound like them — useful for dubbing and accessibility, and abused in scams. and audio deepfakesdeepfake, n.: A fake but realistic video, photo or voice recording made with AI that puts words in a real person's mouth or shows them in a scene that never happened., raising concerns about consent and fraud.
Switching the screen reader to the neural voice made long documents much less tiring to listen to.
Origin
Mechanical speaking machines date to the 18th century, and Homer Dudley's keyboard-played Voder was shown at the 1939 New York World's Fair. The first general English text-to-speech system was built in 1968 by Noriko Umeda and colleagues at Japan's Electrotechnical Laboratory; Dennis Klatt's MITalk and DECtalk brought synthesis into wide use in the 1980s.
Usage note
Text-to-speech generates speech; speech recognitionspeech recognition, n.: Technology that turns spoken words into written text, as when a phone transcribes a voice message or captions appear automatically under a video. transcribes it. Some laws, including the EU AI ActEU AI Act, prop. n.: The European Union's law on artificial intelligence, the first broad AI law anywhere: it bans some uses, sets strict checks for risky ones and puts duties on the makers of the biggest AI models.'s transparency obligationstransparency obligations, n.: EU rules requiring that people be told when they are dealing with AI: chatbots must say they are bots, AI-made content must be marked, and deepfakes must be labeled., require synthetic audio to be marked or disclosed as artificial in certain settings.
Related
- vocodervocoder, n.: A device or program that turns a compact description of a voice back into actual sound; in modern speech AI, the final stage that produces the audio you hear.
- WaveNetWaveNet, prop. n.: A DeepMind model that generates speech and other sounds one tiny audio sample at a time, producing voices far more natural than earlier computer speech.
- mel spectrogrammel spectrogram, n.: A version of a sound picture whose pitch axis is squeezed to match human hearing, giving more detail to low sounds than to high ones; the usual way AI models take in audio.
- voice cloningvoice cloning, n.: Using AI to copy a specific person's voice from recordings, so a computer can say new things that sound like them — useful for dubbing and accessibility, and abused in scams.
- neural audio codecneural audio codec, n.: A neural network that squeezes sound into a very compact stream of numbers and rebuilds it, used both to send audio cheaply and to let AI models 'write' speech and music piece by piece.
- speech-to-speech modelspeech-to-speech model, n.: An AI model that listens to speech and replies or translates directly in speech, without first writing everything down as text, so conversations feel faster and keep the tone of voice.
- mean opinion scoremean opinion score, n.: A quality score from 1 (bad) to 5 (excellent) obtained by asking many listeners to rate audio, such as a synthetic voice or a phone call, and averaging their ratings.
- speech recognitionspeech recognition, n.: Technology that turns spoken words into written text, as when a phone transcribes a voice message or captions appear automatically under a video.
Sources
- van den Oord, A. et al. (2016). WaveNet: A Generative Model for Raw Audio. arXiv:1609.03499. arxiv.org/abs/1609.03499
- Shen, J. et al. (2018). Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions. ICASSP 2018.
- Wang, C. et al. (2023). Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers. arXiv preprint.
Mentioned in
speech recognitionspeech recognition, n.: Technology that turns spoken words into written text, as when a phone transcribes a voice message or captions appear automatically under a video., voice cloningvoice cloning, n.: Using AI to copy a specific person's voice from recordings, so a computer can say new things that sound like them — useful for dubbing and accessibility, and abused in scams., audio language modelaudio language model, n.: An AI language model that can take in sound, such as speech, music and everyday noises, and understand or generate it, much as text chatbots handle words., conversational AIconversational AI, n.: The technology that lets computers hold conversations with people in everyday language, by text or voice. It powers chatbots, voice assistants and automated customer-service lines., mel spectrogrammel spectrogram, n.: A version of a sound picture whose pitch axis is squeezed to match human hearing, giving more detail to low sounds than to high ones; the usual way AI models take in audio., music generationmusic generation, n.: Using AI to compose or produce music — from melodies written as notes to complete songs with vocals — often from a short text description like 'upbeat jazz with saxophone'., Ray KurzweilRay Kurzweil, prop. n.: An American inventor of reading machines and speech technology who became the best-known prophet of the 'singularity', predicting human-level AI by 2029., spectrogramspectrogram, n.: A picture of sound that shows time from left to right, pitch from bottom to top, and loudness as brightness or color, so that speech and music can be seen as patterns., speech-to-speech modelspeech-to-speech model, n.: An AI model that listens to speech and replies or translates directly in speech, without first writing everything down as text, so conversations feel faster and keep the tone of voice., streamingstreaming, n.: Sending an AI's answer piece by piece while it is still being written, so the words appear on screen one after another instead of all at once at the end., virtual assistantvirtual assistant, n.: Software you speak or type to, like Siri, Alexa or Google Assistant, that answers questions and does simple jobs such as setting timers, playing music or controlling smart-home devices., voice agentvoice agent, n.: An AI system you talk to out loud, for example on a phone call, that listens, answers in a natural-sounding voice and can carry out tasks such as booking appointments or checking an order., WaveNetWaveNet, prop. n.: A DeepMind model that generates speech and other sounds one tiny audio sample at a time, producing voices far more natural than earlier computer speech., mean opinion scoremean opinion score, n.: A quality score from 1 (bad) to 5 (excellent) obtained by asking many listeners to rate audio, such as a synthetic voice or a phone call, and averaging their ratings., vocodervocoder, n.: A device or program that turns a compact description of a voice back into actual sound; in modern speech AI, the final stage that produces the audio you hear.