harness
In plain wordsThe software around an AI model that gives it instructions and tools, carries out its actions and decides when it stops. The model is the engine; the harness is the rest of the car.
- 1
Agentsemerging
The runtime that turns a large language modellarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code. into an AI agentAI agent, n.: An AI system that does not just answer but acts: given a goal, it decides on steps, uses tools such as a browser or code editor, checks the results and keeps going until the job is done.. A harness assembles the model's contextcontext window, n.: How much an AI model can read and keep in view at once: the instructions, the conversation so far and any attached documents. Text that does not fit has to be cut, summarized or left out. (system prompt, instructions, tooltool, n.: Any outside capability an AI model is allowed to use — a web search, a calculator, a code runner, a database or an app — described to the model so it can ask for it by name. definitions, files, memory), runs the agent loopagent loop, n.: The repeating cycle at the heart of an AI agent: the model decides what to do next, software carries it out, the result is shown to the model, and the cycle repeats until the task is finished. — calling the model, executing the tool callstool call, n.: A single request from an AI model to use one of its tools — naming the tool and filling in the details — which the surrounding software then carries out and reports back on. it requests and feeding the results back — and enforces stop conditions, permissions, sandboxingsandbox, n.: A sealed-off computer space where an AI agent can run programs and change files without being able to harm the real system, reach private data or contact the outside world unless allowed., error handling and context compactioncompaction, n.: Like replacing a stack of rough notes with a one-page digest: when an AI's conversation grows too long for its memory, the history is condensed into a summary and the work continues from there.. Practitioners summarize the idea as 'agent = model + harness'. Because the same model can perform very differently in two harnesses, agent benchmarks increasingly report the harness alongside the model, and designing it well is called harness engineering.
We kept the model fixed and rewrote the harness, and task success nearly doubled.
- 2
Evaluation
Short for evaluation harnessevaluation harness, n.: Software that puts AI models through tests in a fixed, repeatable way — feeding in the questions, collecting the answers and scoring them — so that every model is judged identically.: software that runs models against benchmarksbenchmark, n.: A fixed, shared test — a set of questions or tasks with an agreed way of scoring — used to compare AI systems with one another and to track progress over time. in a standardized way — formatting prompts, sampling outputs and scoring them — so that results are comparable across models and laboratories.
- 3
Software testing
Short for test harness: code and data that run a program or component under controlled conditions, supply its inputs and check its outputs automatically. This older engineering sense is the source of the other two.
Origin
The lineage runs from horse tack to the software test harness, then to machine-learning evaluation harnesses such as EleutherAI's lm-evaluation-harness (c. 2021), and finally to agent runtimes. The agent sense spread in 2024–2025 as coding agents such as Claude Code and Codex CLI were described as harnesses around a model; by 2026 'harness engineering' was an established practice and the subject of attempts at a formal definition.
Usage note
Usage varies. Broadly, the harness is everything that is not the model; narrowly, it is only the execution loop, while the prompts, tool descriptions and output formats are the scaffoldingscaffolding, n.: The code, prompts and tools built around an AI model so it can carry out tasks it could not manage from a bare prompt — like the scaffolding that lets builders reach every part of a building.. Not to be confused with an agent frameworkagent framework, n.: A software library that gives developers ready-made parts for building AI agents — model connections, tool handling, memory, multi-agent coordination and logging — so they need not write everything from scratch. or SDK, which are libraries for building harnesses.
Related
- scaffoldingscaffolding, n.: The code, prompts and tools built around an AI model so it can carry out tasks it could not manage from a bare prompt — like the scaffolding that lets builders reach every part of a building.
- agent loopagent loop, n.: The repeating cycle at the heart of an AI agent: the model decides what to do next, software carries it out, the result is shown to the model, and the cycle repeats until the task is finished.
- AI agentAI agent, n.: An AI system that does not just answer but acts: given a goal, it decides on steps, uses tools such as a browser or code editor, checks the results and keeps going until the job is done.
- tool usetool use, n.: An AI model's ability to reach beyond its own knowledge by operating outside tools — searching the web, running code, using a calculator or an app — and folding the results into its work.
- context engineeringcontext engineering, n.: Managing an AI model's reading material: choosing, trimming and arranging the instructions, documents, tool results and memories it is given on every turn, so that it has what the task needs and little else.
- evaluation harnessevaluation harness, n.: Software that puts AI models through tests in a fixed, repeatable way — feeding in the questions, collecting the answers and scoring them — so that every model is judged identically.
- subagentsubagent, n.: A helper AI agent that a main agent starts to handle one piece of a job; it works in its own separate workspace and reports back a short summary when it is done.
- modelmodel, n.: A simplified working version of something. In AI it usually means the trained program, a big set of learned numbers, that takes an input and produces an answer, prediction or piece of content.
Sources
- Paniego, S. & Roy Gosthipaty, A. (2026). Harness, Scaffold, and the AI Agent Terms Worth Getting Right. Hugging Face blog. huggingface.co/blog/agent-glossary
- Oliveira de Macedo, S. (2026). What makes a harness a harness: necessary and sufficient conditions for an agent harness. arXiv:2606.10106. arxiv.org/abs/2606.10106
Mentioned in
AI agentAI agent, n.: An AI system that does not just answer but acts: given a goal, it decides on steps, uses tools such as a browser or code editor, checks the results and keeps going until the job is done., computer usecomputer use, n.: An AI's ability to work a computer through its normal screen and controls: it studies pictures of what is displayed and sends mouse and keyboard actions, so it can use ordinary apps that were built for people., evaluationevaluation, n.: Testing an AI system to find out what it can and cannot do, how well, and how safely — from exam-style quizzes to hands-on trials and expert reviews., modelmodel, n.: A simplified working version of something. In AI it usually means the trained program, a big set of learned numbers, that takes an input and produces an answer, prediction or piece of content., SWE-benchSWE-bench, prop. n.: A test that asks AI to fix real bugs and add features in real open-source Python projects, checked by running each project's own tests; the best-known yardstick for AI coding agents., agentagent, n.: Something that acts. In AI, a program that observes its situation and takes actions toward a goal: a game-playing learner, a software helper, or a chatbot that uses tools to complete tasks., agent frameworkagent framework, n.: A software library that gives developers ready-made parts for building AI agents — model connections, tool handling, memory, multi-agent coordination and logging — so they need not write everything from scratch., agent loopagent loop, n.: The repeating cycle at the heart of an AI agent: the model decides what to do next, software carries it out, the result is shown to the model, and the cycle repeats until the task is finished., AI engineeringAI engineering, n.: The craft of building useful, reliable products on top of AI models — choosing models, writing prompts, connecting data and tools, testing quality and running it all safely at scale., Claude CodeClaude Code, prop. n.: A coding assistant from Anthropic that works as an agent: given a programming task, it explores the project, changes the code and runs programs to check its work, from the web, a desktop app, an editor or a terminal., compactioncompaction, n.: Like replacing a stack of rough notes with a one-page digest: when an AI's conversation grows too long for its memory, the history is condensed into a summary and the work continues from there., context engineeringcontext engineering, n.: Managing an AI model's reading material: choosing, trimming and arranging the instructions, documents, tool results and memories it is given on every turn, so that it has what the task needs and little else., evaluation harnessevaluation harness, n.: Software that puts AI models through tests in a fixed, repeatable way — feeding in the questions, collecting the answers and scoring them — so that every model is judged identically., golden datasetgolden dataset, n.: A small, carefully checked set of example questions with known good answers, used as a trusted yardstick to test an AI system every time it changes., orchestrationorchestration, n.: Coordinating several AI calls, tools or agents so each does its part in the right order and their results are combined into one finished piece of work, as a conductor directs an orchestra., permission systempermission system, n.: The rules that decide what an AI agent may do on its own, what it must ask a person about first, and what it may never do — like a phone's settings for which apps can use the camera., scaffoldingscaffolding, n.: The code, prompts and tools built around an AI model so it can carry out tasks it could not manage from a bare prompt — like the scaffolding that lets builders reach every part of a building., scaling wallscaling wall, n.: The disputed idea that making AI models bigger is running out of steam, so progress will soon stall; 'hitting a wall' became the skeptics' slogan and 'there is no wall' the reply., steeringsteering, n.: Guiding what an AI model does — its topic, tone or behavior — while it runs, by changing its instructions, its internal signals or how its next word is chosen, rather than by retraining it., subagentsubagent, n.: A helper AI agent that a main agent starts to handle one piece of a job; it works in its own separate workspace and reports back a short summary when it is done., Terminal-BenchTerminal-Bench, prop. n.: A test of AI agents working in a computer's command line — building software, analyzing data, securing or repairing systems — with each task checked automatically inside a sealed-off container., The Last DictionaryThe Last Dictionary, prop. n.: This book: a dictionary of intelligence, human and otherwise, which explains each word first in plain language, then precisely, and says when it was born., tooltool, n.: Any outside capability an AI model is allowed to use — a web search, a calculator, a code runner, a database or an app — described to the model so it can ask for it by name., tool calltool call, n.: A single request from an AI model to use one of its tools — naming the tool and filling in the details — which the surrounding software then carries out and reports back on.