evaluation
In plain wordsTesting an AI system to find out what it can and cannot do, how well, and how safely — from exam-style quizzes to hands-on trials and expert reviews.
- 1
General
The systematic measurement of an AI system's behavior against a stated purpose, producing evidence about its capabilities, reliability or risks. An evaluation specifies what is measured (the construct), the inputs or tasks, how the system is run (prompt format, harnessharness, n.: The software around an AI model that gives it instructions and tools, carries out its actions and decides when it stops. The model is the engine; the harness is the rest of the car., tools, sampling settings, number of attempts) and how outputs are scored — by exact rules, unit tests, a model graderLLM-as-a-judge, n.: Using one AI language model to grade or compare the answers of another, as a cheaper and faster stand-in for human reviewers. or human ratershuman evaluation, n.: Judging an AI's outputs by asking people to rate or compare them, because for many tasks — writing, conversation, translation — no automatic score captures quality well.. Families include static benchmarksbenchmark, n.: A fixed, shared test — a set of questions or tasks with an agreed way of scoring — used to compare AI systems with one another and to track progress over time., preference comparisons such as LMArenaLMArena, prop. n.: A public website that ranks AI chatbots by blind taste test: visitors put one question to two unnamed models, pick the reply they prefer, and millions of such votes produce the rankings., agentic task suites, red teamingred teaming, n.: Deliberately attacking your own AI system — trying to make it misbehave, leak secrets or help with harm — so that weaknesses are found before real adversaries or users find them., dangerous-capability evaluationsdangerous capability evaluation, n.: A test of whether an AI system can do things that could cause severe harm in the wrong hands or on its own — such as helping make a bioweapon, hacking computers or copying itself. and post-deployment monitoring. Offline evaluation tests a model before release; online evaluation measures it in use, for example by A/B testingA/B testing, n.: Comparing two versions of something — two models, two prompts, two designs — by randomly showing each to different groups of real users and measuring which works better.. Because results depend heavily on setup, credible reports state the configuration and uncertainty and guard against data contaminationdata contamination, n.: When the test questions used to grade an AI system were already in its training data, so a high score may reflect memorized answers rather than real ability. and failures of construct validityconstruct validity, n.: Whether high marks on a test actually reflect the skill the test is named after — for example, whether a 'reasoning' test rewards reasoning rather than memory or test-taking tricks..
Before shipping the new model we ran the full eval suite, including the internal agentic tasks.
- 2
Informal (an eval)
A single test or test suite — a dataset, set of scenarios or grading script — used repeatedly to check a model or AI product; plural 'evals'. Among practitioners, 'writing evals' means building the cases and graders that define success for an application, the basis of evals-driven developmentevals-driven development, n.: Building an AI product by first writing tests that show what good answers look like, then accepting changes to prompts, models or code only when the test scores improve..
Origin
Evaluation has been central to AI since the Turing testTuring test, n.: Alan Turing's proposed test of machine intelligence: a judge holds typed conversations with a hidden person and a hidden computer, and the computer passes if the judge cannot consistently pick it out. (1950) and the human ratings of machine translation in the 1966 ALPAC reportALPAC report, prop. n.: A 1966 US government report that concluded machine translation was slower, worse and costlier than human translators. It ended most American funding for the field for many years.. Shared tasks funded by DARPA, such as the Message Understanding Conferences (from 1987), and NIST's TREC (from 1992) made common test sets and metrics the field's norm. The clipped form 'evals' spread with large language models, helped by OpenAI's open-source Evals framework (March 2023).
Usage note
An evaluation shows what a system does under test conditions; it does not by itself establish behavior in deployment. Model evaluation differs from monitoring a validation setvalidation set, n.: A set of examples kept out of training and used during development to compare model versions and settings, like a practice exam taken before the real one. during training, and from an AI auditAI audit, n.: An independent check of an AI system, or of the company using it, against agreed standards — for example, testing whether a hiring tool treats women and men fairly., which also examines an organization's processes.
Related
- benchmarkbenchmark, n.: A fixed, shared test — a set of questions or tasks with an agreed way of scoring — used to compare AI systems with one another and to track progress over time.
- evaluation harnessevaluation harness, n.: Software that puts AI models through tests in a fixed, repeatable way — feeding in the questions, collecting the answers and scoring them — so that every model is judged identically.
- human evaluationhuman evaluation, n.: Judging an AI's outputs by asking people to rate or compare them, because for many tasks — writing, conversation, translation — no automatic score captures quality well.
- LLM-as-a-judgeLLM-as-a-judge, n.: Using one AI language model to grade or compare the answers of another, as a cheaper and faster stand-in for human reviewers.
- construct validityconstruct validity, n.: Whether high marks on a test actually reflect the skill the test is named after — for example, whether a 'reasoning' test rewards reasoning rather than memory or test-taking tricks.
- dangerous capability evaluationdangerous capability evaluation, n.: A test of whether an AI system can do things that could cause severe harm in the wrong hands or on its own — such as helping make a bioweapon, hacking computers or copying itself.
- third-party AI evaluationthird-party AI evaluation, n.: Testing of an AI system by someone other than the company that made it — a government institute, a nonprofit lab or independent researchers — to check what it can do and how risky it is.
- evals-driven developmentevals-driven development, n.: Building an AI product by first writing tests that show what good answers look like, then accepting changes to prompts, models or code only when the test scores improve.
- AI auditAI audit, n.: An independent check of an AI system, or of the company using it, against agreed standards — for example, testing whether a hiring tool treats women and men fairly.
Mentioned in
benchmarkbenchmark, n.: A fixed, shared test — a set of questions or tasks with an agreed way of scoring — used to compare AI systems with one another and to track progress over time., regressionregression, n.: Predicting a number rather than a category — a house's price, tomorrow's temperature, a patient's blood pressure — by learning from past examples where the answer is known., A/B testingA/B testing, n.: Comparing two versions of something — two models, two prompts, two designs — by randomly showing each to different groups of real users and measuring which works better., accidental cyberattackaccidental cyberattack, n.: A break-in on real computer systems carried out by AI agents that were only supposed to be in training or being tested, with no person meaning to attack anyone., AI deceptionAI deception, n.: When an AI system leads people to believe something false in order to achieve some goal — by lying, bluffing or hiding what it is really doing., AI engineeringAI engineering, n.: The craft of building useful, reliable products on top of AI models — choosing models, writing prompts, connecting data and tools, testing quality and running it all safely at scale., backdoor attackbackdoor attack, n.: Secretly training an AI model to misbehave whenever it sees a hidden signal — a sticker, a word, a date — while it acts perfectly normally the rest of the time., baselinebaseline, n.: A simple reference method that a new model must beat to show it is worth anything, such as always guessing the most common answer or reusing yesterday's value., capabilitycapability, n.: Something an AI system is able to do, such as writing working code or solving math problems, and how well. Measuring capabilities is how researchers track progress and spot dangerous skills., construct validityconstruct validity, n.: Whether high marks on a test actually reflect the skill the test is named after — for example, whether a 'reasoning' test rewards reasoning rather than memory or test-taking tricks., deceptive alignmentdeceptive alignment, n.: A feared failure in which an AI with hidden goals behaves well during training and testing only because it knows it is being watched, planning to act differently later., evaluation awarenessevaluation awareness, n.: When an AI model can tell it is being tested rather than used for real — which may change how it behaves and make test results less trustworthy., evaluation harnessevaluation harness, n.: Software that puts AI models through tests in a fixed, repeatable way — feeding in the questions, collecting the answers and scoring them — so that every model is judged identically., golden datasetgolden dataset, n.: A small, carefully checked set of example questions with known good answers, used as a trusted yardstick to test an AI system every time it changes., ground truthground truth, n.: The reference answers treated as correct when checking or training an AI system — for example, the confirmed diagnoses that a medical-imaging model's guesses are compared against., mean reciprocal rankmean reciprocal rank, n.: A score for search: for each question, note how far down the first correct answer appears, take one over that position, and average across questions. First place scores 1, second place 0.5., model cardmodel card, n.: A short document published with an AI model explaining what it is for, how it was trained and tested, how well it works for different groups of people, and its known limits., normalized discounted cumulative gainnormalized discounted cumulative gain, n.: A score for a ranked list of results that rewards putting the most relevant items near the top, scaled so that a perfect ordering scores 1., prompt sensitivityprompt sensitivity, n.: The way an AI's answers, and its test scores, can change a lot after small changes to a prompt that should not matter — a different word, an extra space or examples in another order., recall@krecall@k, n.: A score for search systems: of all the right answers that exist, what fraction showed up in the top k results? Higher means fewer relevant items were missed., sandbaggingsandbagging, n.: When an AI deliberately performs worse than it can — for example on a safety test — so that it seems less capable or less dangerous than it really is., task (machine learning)task (machine learning), n.: The job an AI model is meant to do, such as translating sentences, spotting spam or answering questions, defined by its inputs, its expected outputs and how success is scored., dark factorydark factory, n.: A way of building software in which AI agents write, test and ship all the code and no human writes or reads it, like a factory run with the lights off., decision modeldecision model, n.: A kind of AI language model that reads text but answers only with numbers — a yes-or-no probability, a choice or a score — instead of writing words.