Skip to content
Last Dictionary
Plane
Vision, Speech & PerceptionEvaluation & Benchmarks

mean opinion score

nounAlso: MOS, comparative mean opinion score, CMOS, MUSHRA, predicted MOS

In plain wordsA quality score from 1 (bad) to 5 (excellent) obtained by asking many listeners to rate audio, such as a synthetic voice or a phone call, and averaging their ratings.

  1. The average of listeners' ratings of a stimulus, typically speech, on a five-point absolute category scale: 1 bad, 2 poor, 3 fair, 4 good, 5 excellent. Developed for judging telephone transmission quality, it became the standard subjective measure for text-to-speechtext-to-speech, n.: Technology that reads written text aloud in a computer-generated voice, used in screen readers, voice assistants, audiobooks and navigation apps., voice conversion and speech enhancement, usually reported with a 95% confidence interval and alongside the score of real recordings as a ceiling. Variants include comparative MOS (CMOS), in which listeners rate one sample against another, and MUSHRA, which presents several systems and a hidden reference together to resolve finer differences. Because listening tests are slow and costly, objective estimators predict MOS: signal-based telephony standards such as PESQ and POLQA, and neural predictors for synthetic speech. MOS values are not comparable across studies with different listeners, instructions, scales or competing systems, and near-human synthesis has squeezed differences into a narrow band near the top of the scale.

    The new voice scored 4.4 MOS against 4.5 for the studio recordings, inside the confidence interval.

MOS=1N∑n=1NRn\text{MOS} = \frac{1}{N}\sum_{n=1}^{N} R_n

Origin

Arose in telephone engineering, where listener panels rated transmission quality; the procedure was standardized by the International Telecommunication Union in Recommendation P.800 (1996), with MOS terminology set out in P.800.1. Speech-synthesis research adopted it as its main listening test.

Usage note

Here R_n is the n-th of N ratings. Video and imaging use MOS in the same way for perceived visual quality. Results reported without confidence intervals or details of the listener pool are hard to interpret.

Related

See also
  • text-to-speechtext-to-speech, n.: Technology that reads written text aloud in a computer-generated voice, used in screen readers, voice assistants, audiobooks and navigation apps.
  • human evaluationhuman evaluation, n.: Judging an AI's outputs by asking people to rate or compare them, because for many tasks — writing, conversation, translation — no automatic score captures quality well.
  • vocodervocoder, n.: A device or program that turns a compact description of a voice back into actual sound; in modern speech AI, the final stage that produces the audio you hear.
  • evaluationevaluation, n.: Testing an AI system to find out what it can and cannot do, how well, and how safely — from exam-style quizzes to hands-on trials and expert reviews.
  • benchmarkbenchmark, n.: A fixed, shared test — a set of questions or tasks with an agreed way of scoring — used to compare AI systems with one another and to track progress over time.

Sources

  1. ITU-T (1996). Recommendation P.800: Methods for Subjective Determination of Transmission Quality. International Telecommunication Union.

Mentioned in

text-to-speechtext-to-speech, n.: Technology that reads written text aloud in a computer-generated voice, used in screen readers, voice assistants, audiobooks and navigation apps.