JSON mode
In plain wordsA setting on some AI services that makes the model reply only in JSON, a standard text format for data, so that programs can read the reply reliably.
An API option that constrains a language model's output to be syntactically valid JSON (JavaScript Object Notation), the text format of nested objects, arrays, strings, numbers and Booleans used throughout web software. It is implemented by constrained decodingconstrained decoding, n.: Forcing an AI model's output to follow a required format — such as valid code or a fill-in form — by blocking, at every step, any word that would break the rules.: at each step the server permits only tokenstoken, n.: The small chunk of text an AI language model reads and writes — often a word, part of a word or a punctuation mark. AI usage, limits and prices are counted in tokens. that keep the output parseable, so responses are not wrapped in prose or left with unclosed brackets, except when the length limit cuts them off. JSON mode does not fix which keys, types or values appear, so developers still describe the desired shape in the prompt and validate the result; guarantees at the level of a schema require structured outputsstructured outputs, n.: Getting an AI model to answer in an exact, machine-readable format — like a form with fixed fields — so other software can use the answer without guesswork.. OpenAI's version rejects requests whose prompt never mentions JSON, since a model forced into JSON without having been asked for it can pad its reply with whitespace until the length limit.
Turn on JSON mode, but keep validating: it guarantees parseable output, not the right fields.
Origin
Introduced by OpenAIOpenAI, prop. n.: The American company that makes ChatGPT and the GPT models. It began in 2015 as a nonprofit research lab and is now a for-profit company overseen by a nonprofit foundation. at its DevDay event in November 2023 as the response_format option of its Chat Completions API. Open-source engines had or soon added grammar-based equivalents (llama.cpp added grammar-constrained sampling in mid-2023), and schema-enforcing structured outputsstructured outputs, n.: Getting an AI model to answer in an exact, machine-readable format — like a form with fixed fields — so other software can use the answer without guesswork. largely superseded JSON mode from August 2024.
Usage note
Different from simply asking a model for JSON in the prompt, which can fail by adding commentary or malformed syntax. JSON itself was specified by Douglas Crockford in the early 2000s.
Related
- structured outputsstructured outputs, n.: Getting an AI model to answer in an exact, machine-readable format — like a form with fixed fields — so other software can use the answer without guesswork.
- JSON SchemaJSON Schema, prop. n.: A standard way of writing down exactly what a piece of JSON data must look like — which fields, of which types — so that software and AI models can check it or follow it.
- constrained decodingconstrained decoding, n.: Forcing an AI model's output to follow a required format — such as valid code or a fill-in form — by blocking, at every step, any word that would break the rules.
- model APImodel API, n.: A web service through which programs send requests to an AI model run by a company and receive its answers, usually paying per use instead of running the model themselves.
- function callingfunction calling, n.: A feature of AI model services that lets the model reply with a neatly formatted request to run one of a program's functions, rather than with ordinary text; the program, not the model, then runs it.
- structured outputsstructured outputs, n.: Getting an AI model to answer in an exact, machine-readable format — like a form with fixed fields — so other software can use the answer without guesswork.
Mentioned in
constrained decodingconstrained decoding, n.: Forcing an AI model's output to follow a required format — such as valid code or a fill-in form — by blocking, at every step, any word that would break the rules., function callingfunction calling, n.: A feature of AI model services that lets the model reply with a neatly formatted request to run one of a program's functions, rather than with ordinary text; the program, not the model, then runs it., JSON SchemaJSON Schema, prop. n.: A standard way of writing down exactly what a piece of JSON data must look like — which fields, of which types — so that software and AI models can check it or follow it., structured outputsstructured outputs, n.: Getting an AI model to answer in an exact, machine-readable format — like a form with fixed fields — so other software can use the answer without guesswork.