structured outputs
In plain wordsGetting an AI model to answer in an exact, machine-readable format — like a form with fixed fields — so other software can use the answer without guesswork.
Model output that conforms to a developer-specified schema — typically a JSON SchemaJSON Schema, prop. n.: A standard way of writing down exactly what a piece of JSON data must look like — which fields, of which types — so that software and AI models can check it or follow it. or a type declared in code, such as a Pydantic model or a TypeScript type — so that software can parse and validate it reliably. Approaches differ in strictness: prompting with format instructions and examples, then validating and retrying on failure (as libraries such as Instructor automate); JSON modeJSON mode, n.: A setting on some AI services that makes the model reply only in JSON, a standard text format for data, so that programs can read the reply reliably., which guarantees syntactically valid JSON but not a particular shape; and schema enforcement through constrained decodingconstrained decoding, n.: Forcing an AI model's output to follow a required format — such as valid code or a fill-in form — by blocking, at every step, any word that would break the rules., in which the inferenceinference, n.: Drawing a conclusion from evidence. In today's AI it most often means running an already-trained model to get an answer, as opposed to training it. engine masks every tokentoken, n.: The small chunk of text an AI language model reads and writes — often a word, part of a word or a punctuation mark. AI usage, limits and prices are counted in tokens. that would violate the schema, guaranteeing conformance. Structured outputs underpin data extraction, classification, tool callstool call, n.: A single request from an AI model to use one of its tools — naming the tool and filling in the details — which the surrounding software then carries out and reports back on. and multi-step pipelines. Enforcement guarantees form, not truth: a model forced into a schema may fill fields with plausible but wrong values, and very rigid formats can degrade reasoning unless the schema leaves room for it, for example with a field for working before the answer.
With structured outputs on, every invoice extraction comes back with the same twelve fields.
Origin
Developers first coaxed formats from models with prompts and parsers, then with constrained-generation libraries such as Microsoft's Guidance and Outlines (2023). OpenAI's JSON modeJSON mode, n.: A setting on some AI services that makes the model reply only in JSON, a standard text format for data, so that programs can read the reply reliably. (November 2023) guaranteed valid JSON, and its 'Structured Outputs' feature (August 2024) guaranteed adherence to a supplied schema; other providers and open-source servers adopted the capability and the name.
Usage note
Closely related to function callingfunction calling, n.: A feature of AI model services that lets the model reply with a neatly formatted request to run one of a program's functions, rather than with ordinary text; the program, not the model, then runs it., where the schema describes a tool's arguments; both usually rely on the same constrained-decoding machinery. Not to be confused with structured prediction, the older machine-learning problem of predicting structured objects such as parse trees.
Related
- JSON modeJSON mode, n.: A setting on some AI services that makes the model reply only in JSON, a standard text format for data, so that programs can read the reply reliably.
- JSON SchemaJSON Schema, prop. n.: A standard way of writing down exactly what a piece of JSON data must look like — which fields, of which types — so that software and AI models can check it or follow it.
- constrained decodingconstrained decoding, n.: Forcing an AI model's output to follow a required format — such as valid code or a fill-in form — by blocking, at every step, any word that would break the rules.
- function callingfunction calling, n.: A feature of AI model services that lets the model reply with a neatly formatted request to run one of a program's functions, rather than with ordinary text; the program, not the model, then runs it.
- tool calltool call, n.: A single request from an AI model to use one of its tools — naming the tool and filling in the details — which the surrounding software then carries out and reports back on.
- model APImodel API, n.: A web service through which programs send requests to an AI model run by a company and receive its answers, usually paying per use instead of running the model themselves.
Sources
- Willard, B. T. & Louf, R. (2023). Efficient Guided Generation for Large Language Models. arXiv:2307.09702. arxiv.org/abs/2307.09702
Mentioned in
model APImodel API, n.: A web service through which programs send requests to an AI model run by a company and receive its answers, usually paying per use instead of running the model themselves., AI engineeringAI engineering, n.: The craft of building useful, reliable products on top of AI models — choosing models, writing prompts, connecting data and tools, testing quality and running it all safely at scale., constrained decodingconstrained decoding, n.: Forcing an AI model's output to follow a required format — such as valid code or a fill-in form — by blocking, at every step, any word that would break the rules., document understandingdocument understanding, n.: Using AI to read documents such as invoices, forms, contracts and scientific papers the way a person does, taking in layout, tables and charts as well as words, to pull out the information needed., function callingfunction calling, n.: A feature of AI model services that lets the model reply with a neatly formatted request to run one of a program's functions, rather than with ordinary text; the program, not the model, then runs it., information extractioninformation extraction, n.: Automatically pulling structured facts out of ordinary text — who did what, when, where and for how much — and putting them into a table or database., JSON SchemaJSON Schema, prop. n.: A standard way of writing down exactly what a piece of JSON data must look like — which fields, of which types — so that software and AI models can check it or follow it., OpenAI-compatible APIOpenAI-compatible API, n.: An AI service that accepts the same request format as OpenAI's, so software written for OpenAI's models can use other companies' or self-hosted models by changing an address and key., streamingstreaming, n.: Sending an AI's answer piece by piece while it is still being written, so the words appear on screen one after another instead of all at once at the end., decision modeldecision model, n.: A kind of AI language model that reads text but answers only with numbers — a yes-or-no probability, a choice or a score — instead of writing words., IFEvalIFEval, prop. n.: A test of whether AI models obey precise formatting and content rules — such as 'use exactly three bullet points' or 'answer in capital letters' — that a computer program can check automatically., JSON modeJSON mode, n.: A setting on some AI services that makes the model reply only in JSON, a standard text format for data, so that programs can read the reply reliably., SGLangSGLang, prop. n.: A free, open-source engine for running AI language and image models on servers, built to be fast when many requests begin the same way, as in chat and agent workloads.