LLM-as-a-judge
In plain wordsUsing one AI language model to grade or compare the answers of another, as a cheaper and faster stand-in for human reviewers.
An evaluation method in which a large language modellarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code. scores outputs — on a rating scale, against a reference answer or rubric, or by choosing the better of two responses — instead of or alongside human ratershuman evaluation, n.: Judging an AI's outputs by asking people to rate or compare them, because for many tasks — writing, conversation, translation — no automatic score captures quality well.. Judges make open-ended outputs gradable at scale and underpin benchmarks such as MT-BenchMT-Bench, prop. n.: A set of 80 two-part questions — in writing, reasoning, math, coding and more — whose answers are graded by a strong AI model, used to compare chat assistants quickly. and AlpacaEval; the same idea supplies training signal in reinforcement learning from AI feedbackreinforcement learning from AI feedback, n.: Training an AI with preference judgments made by another AI instead of by people, which is cheaper and faster but only as good as the AI doing the judging. and rubric-based rewardsrubric reward, n.: A training score built from a checklist of specific things a good answer should do, with an AI grader ticking off each one — a way to reward quality where no single right answer exists.. Documented biases include position bias (favoring the answer shown first or second), verbosity bias (favoring longer answers), self-preference (favoring outputs from the judge's own model family) and susceptibility to confident tone and formatting. Mitigations include swapping answer order, controlling for length, expert-written rubrics, panels of judges, and periodic calibration against human labels.
We used an LLM-as-a-judge with a twelve-point rubric and spot-checked 200 verdicts against physicians' ratings.
Origin
Language models were used to score generated text in early 2023 (GPTScore, G-Eval), and the Vicuna team graded chatbots with GPT-4 that spring. The phrase was popularized by Lianmin Zheng and colleagues' June 2023 paper 'Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena', which found GPT-4's verdicts agreed with human preferences over 80% of the time, about as often as humans agree with each other.
Usage note
Good average agreement with humans does not guarantee reliability on hard or adversarial items, and outputs optimized against a fixed judge can exploit it — a form of reward hackingreward hacking, n.: When an AI finds a shortcut that earns the reward or score it is trained on without doing the task its designers wanted — like a student gaming the grading rubric..
Related
- human evaluationhuman evaluation, n.: Judging an AI's outputs by asking people to rate or compare them, because for many tasks — writing, conversation, translation — no automatic score captures quality well.
- MT-BenchMT-Bench, prop. n.: A set of 80 two-part questions — in writing, reasoning, math, coding and more — whose answers are graded by a strong AI model, used to compare chat assistants quickly.
- win ratewin rate, n.: The share of head-to-head comparisons in which judges — people or another AI — prefer one model's answer over a rival's for the same request.
- reinforcement learning from AI feedbackreinforcement learning from AI feedback, n.: Training an AI with preference judgments made by another AI instead of by people, which is cheaper and faster but only as good as the AI doing the judging.
- rubric rewardrubric reward, n.: A training score built from a checklist of specific things a good answer should do, with an AI grader ticking off each one — a way to reward quality where no single right answer exists.
- reward modelreward model, n.: A separate AI model trained to score responses the way people would, so it can stand in for human judges while another model is trained on millions of examples.
- verifierverifier, n.: A program or AI model that checks whether a proposed answer is correct — by running tests, comparing with a known result or judging the reasoning — so that good answers can be picked or rewarded.
- exact matchexact match, n.: The simplest way to grade an AI's answer: it counts as right only if it is identical to the reference answer, after minor tidying such as ignoring capital letters.
Sources
- Zheng, L., Chiang, W.-L., Sheng, Y. et al. (2023). Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. NeurIPS 2023 Datasets and Benchmarks Track. arxiv.org/abs/2306.05685
Mentioned in
correlationcorrelation, n.: A measure of how closely two quantities move together, from −1 (one rises as the other falls) through 0 (no straight-line link) to +1 (they rise together). It does not show that one causes the other., evaluationevaluation, n.: Testing an AI system to find out what it can and cannot do, how well, and how safely — from exam-style quizzes to hands-on trials and expert reviews., LMArenaLMArena, prop. n.: A public website that ranks AI chatbots by blind taste test: visitors put one question to two unnamed models, pick the reply they prefer, and millions of such votes produce the rankings., annotationannotation, n.: Adding information to raw data by hand or with software — marking objects in photos, tagging names in sentences, transcribing speech or ranking chatbot answers — so machines can learn from it., automated red teamingautomated red teaming, n.: Using AI programs, not just human testers, to invent huge numbers of tricky inputs that try to make another AI misbehave, so problems can be found and fixed at scale., biasbias, n.: A word with several AI meanings: an adjustable number added inside a model; a systematic error in estimates; or unfair treatment of some groups of people by an AI system., citationcitation, n.: A pointer from something an AI says to the source that supports it — a link, document or quoted passage — so people can check the claim for themselves., exact matchexact match, n.: The simplest way to grade an AI's answer: it counts as right only if it is identical to the reference answer, after minor tidying such as ignoring capital letters., golden datasetgolden dataset, n.: A small, carefully checked set of example questions with known good answers, used as a trusted yardstick to test an AI system every time it changes., ground truthground truth, n.: The reference answers treated as correct when checking or training an AI system — for example, the confirmed diagnoses that a medical-imaging model's guesses are compared against., groundinggrounding, n.: Connecting an AI's words to something real — trusted documents, a live search, images or the physical world — so that its answers rest on evidence rather than only on what it absorbed in training., human evaluationhuman evaluation, n.: Judging an AI's outputs by asking people to rate or compare them, because for many tasks — writing, conversation, translation — no automatic score captures quality well., instruction followinginstruction following, n.: An AI's ability to do exactly what it is asked, including details such as length, format, language and things to avoid, rather than something merely related., instruction-based image editinginstruction-based image editing, n.: Changing a picture by telling an AI what to change — 'make it night', 'remove the car', 'give her a red coat' — instead of using editing tools by hand., natural language generationnatural language generation, n.: Getting computers to produce human language — from turning a spreadsheet into a weather report to writing an essay or a chatbot's reply., reinforcement learning from AI feedbackreinforcement learning from AI feedback, n.: Training an AI with preference judgments made by another AI instead of by people, which is cheaper and faster but only as good as the AI doing the judging., reward modelreward model, n.: A separate AI model trained to score responses the way people would, so it can stand in for human judges while another model is trained on millions of examples., ROUGEROUGE, abbr.: An automatic score for summaries that checks how much of the wording in human-written reference summaries also appears in the computer-generated one., verifierverifier, n.: A program or AI model that checks whether a proposed answer is correct — by running tests, comparing with a known result or judging the reasoning — so that good answers can be picked or rewarded., win ratewin rate, n.: The share of head-to-head comparisons in which judges — people or another AI — prefer one model's answer over a rival's for the same request., automated interpretabilityautomated interpretability, n.: Using AI models to help explain other AI models — for instance, having a language model read examples that excite an internal unit and write a description of what it detects., BERTScoreBERTScore, prop. n.: An automatic score for computer-written text that compares meanings rather than exact words, using an AI language model to judge how closely each word matches the words of a reference text., CLIP scoreCLIP score, n.: A score for how well a picture matches a text description, computed by an AI model that was trained to pair images with their captions., evals-driven developmentevals-driven development, n.: Building an AI product by first writing tests that show what good answers look like, then accepting changes to prompts, models or code only when the test scores improve.