MFCC
In plain wordsA short list of numbers, usually about 13, that summarizes the shape of a brief slice of sound in a way that mimics human hearing; for decades the standard input to speech recognizers.
Mel-frequency cepstral coefficients: compact features describing the short-term spectral envelope of a sound. Each short frame of audio (about 25 milliseconds long, every 10 milliseconds) is windowed and Fourier-transformed; its power spectrum is pooled by a mel-spaced triangular filterbank, as in a mel spectrogrammel spectrogram, n.: A version of a sound picture whose pitch axis is squeezed to match human hearing, giving more detail to low sounds than to high ones; the usual way AI models take in audio.; the logarithms of the filter energies are taken; and a discrete cosine transform is applied, of which typically the first 12 or 13 coefficients are kept. The low-order coefficients capture the broad shape of the spectrum, which reflects the configuration of the vocal tract and hence the phoneme, while discarding fine harmonic detail tied to pitch. The decorrelating cosine transform suited the diagonal-covariance Gaussian mixturesGaussian mixture model, n.: A model that explains data as coming from several overlapping bell-shaped clouds, working out where each cloud sits, how spread out it is, and how likely each point is to belong to it. of hidden Markov modelhidden Markov model, n.: A model of a sequence, such as speech or DNA, in which hidden states — say, the words being spoken — change step by step at random, and each state gives off signals that can be observed, such as sounds. recognizers, which usually appended energy and first and second time derivatives ('deltas') to make a 39-dimensional vector. Deep networks generally take log-mel spectrograms instead, since the transform discards information they can use, but MFCCs remain common in speaker recognition, music information retrieval, classical baselines and low-power devices.
The keyword spotter on the microcontroller still uses 13 MFCCs per frame to save memory.
Origin
The cepstrum (a respelling of 'spectrum') was named by Bruce Bogert, Michael Healy and John Tukey in 1963. Mel-frequency cepstral coefficients were popularized by Steven Davis and Paul Mermelstein's 1980 comparison of speech-recognition features, building on mid-1970s work by Mermelstein and by John Bridle and Michael Brown.
Usage note
Usually used in the plural, MFCCs. 'Cepstrum' and 'quefrency' are deliberate letter-swapped coinages from Bogert, Healy and Tukey's paper.
Related
- mel spectrogrammel spectrogram, n.: A version of a sound picture whose pitch axis is squeezed to match human hearing, giving more detail to low sounds than to high ones; the usual way AI models take in audio.
- spectrogramspectrogram, n.: A picture of sound that shows time from left to right, pitch from bottom to top, and loudness as brightness or color, so that speech and music can be seen as patterns.
- feature extractionfeature extraction, n.: Turning raw data such as pixels or sound into a short list of telling numbers — computed by hand-designed rules or by an AI system trained earlier — that a new model can learn from or compare.
- speech recognitionspeech recognition, n.: Technology that turns spoken words into written text, as when a phone transcribes a voice message or captions appear automatically under a video.
- speaker verificationspeaker verification, n.: Checking whether a voice really belongs to the person it claims to be, as when a bank confirms your identity by the sound of your voice on the phone.
- hidden Markov modelhidden Markov model, n.: A model of a sequence, such as speech or DNA, in which hidden states — say, the words being spoken — change step by step at random, and each state gives off signals that can be observed, such as sounds.
Sources
- Davis, S. B. & Mermelstein, P. (1980). Comparison of Parametric Representations for Monosyllabic Word Recognition in Continuously Spoken Sentences. IEEE Transactions on Acoustics, Speech, and Signal Processing 28(4), 357–366.
- Bogert, B. P., Healy, M. J. R. & Tukey, J. W. (1963). The Quefrency Alanysis of Time Series for Echoes: Cepstrum, Pseudo-Autocovariance, Cross-Cepstrum and Saphe Cracking. In Proceedings of the Symposium on Time Series Analysis, Wiley.
Mentioned in
speech recognitionspeech recognition, n.: Technology that turns spoken words into written text, as when a phone transcribes a voice message or captions appear automatically under a video., feature extractionfeature extraction, n.: Turning raw data such as pixels or sound into a short list of telling numbers — computed by hand-designed rules or by an AI system trained earlier — that a new model can learn from or compare., mel spectrogrammel spectrogram, n.: A version of a sound picture whose pitch axis is squeezed to match human hearing, giving more detail to low sounds than to high ones; the usual way AI models take in audio., speaker verificationspeaker verification, n.: Checking whether a voice really belongs to the person it claims to be, as when a bank confirms your identity by the sound of your voice on the phone., spectrogramspectrogram, n.: A picture of sound that shows time from left to right, pitch from bottom to top, and loudness as brightness or color, so that speech and music can be seen as patterns., wake wordwake word, n.: A word or phrase, such as 'Alexa' or 'Hey Google', that a smart speaker or phone listens for constantly and that switches it into full listening mode.