chat template
In plain wordsThe exact text layout a chat AI was trained on for marking who is speaking — system, user or assistant. Software must reproduce it precisely, or the model performs noticeably worse.
The model-specific formatting rule that flattens a conversation (system instructions, user and assistant turns, tool messages) into the one stream of tokens a chat modelchat model, n.: A language model trained to hold a back-and-forth conversation, keeping track of who said what and replying as a helpful assistant. It is the engine inside chat apps. reads. Each turn boundary is marked with special tokensspecial token, n.: A reserved symbol in an AI model's vocabulary that is not ordinary text but a signal — 'start of message', 'end of answer', 'begin thinking' — used to structure what the model reads and writes. or marker strings — for example ChatML's <|im_start|>user … <|im_end|> or Mistral's [INST] … [/INST] — and the sequence can end with the opening of an assistant turn (a generation prompt) so that the model replies rather than continuing the user's text. Because the model was post-trained in exactly this format, a mismatched template silently degrades quality, breaks stopping or disables tool calling. In the Hugging Face ecosystem the template is a Jinja program stored with the tokenizertokenizer, n.: The component that splits text into tokens and converts them to numbers before an AI model sees it, then turns the model's output numbers back into text. and applied by apply_chat_template; hosted APIs apply theirs server-side. Templates also define how system prompts, tool definitions, tool results and reasoning blocks are rendered.
Outputs were garbage until we noticed the server was applying the Llama chat template to a Qwen model.
Origin
OpenAI described its Chat Markup Language (ChatML) when it opened the ChatGPT model to developers in March 2023. As open chat models multiplied with incompatible formats, Hugging Face added chat templates to its TransformersTransformers, prop. n.: A free software library from Hugging Face that lets programmers download and use thousands of ready-trained AI models for text, images and sound with a few lines of code. library in October 2023, so that each model could ship its own formatting rules as a Jinja template saved with its tokenizer.
Related
- chat modelchat model, n.: A language model trained to hold a back-and-forth conversation, keeping track of who said what and replying as a helpful assistant. It is the engine inside chat apps.
- special tokenspecial token, n.: A reserved symbol in an AI model's vocabulary that is not ordinary text but a signal — 'start of message', 'end of answer', 'begin thinking' — used to structure what the model reads and writes.
- prompt templateprompt template, n.: A reusable prompt with blanks to fill in — for example, 'Summarize this email for a busy manager: {email}' — so a program can send the same tested instructions with different content each time.
- tokenizertokenizer, n.: The component that splits text into tokens and converts them to numbers before an AI model sees it, then turns the model's output numbers back into text.
- system promptsystem prompt, n.: Background instructions given to a chatbot before the conversation starts, setting its role, rules, tone and tools. Users usually never see it, but it shapes every reply.
- stop sequencestop sequence, n.: A piece of text that tells an AI model to stop writing as soon as it produces it — for example, stopping at a blank line or just before the next question.
- TransformersTransformers, prop. n.: A free software library from Hugging Face that lets programmers download and use thousands of ready-trained AI models for text, images and sound with a few lines of code.
- prompt templateprompt template, n.: A reusable prompt with blanks to fill in — for example, 'Summarize this email for a busy manager: {email}' — so a program can send the same tested instructions with different content each time.
Sources
- Hugging Face (2026). Chat templates. Transformers documentation. huggingface.co/docs/transformers/main/en/chat_templating
- OpenAI (2023). Introducing ChatGPT and Whisper APIs. OpenAI blog. openai.com/index/introducing-chatgpt-and-whisper-apis
- Carrigan, M. (2023). Chat Templates: An End to the Silent Performance Killer. Hugging Face blog, 3 October 2023. huggingface.co/blog/chat-templates
Mentioned in
promptprompt, n.: The message or instructions you give an AI system — a question, a request, some examples or a document — that it responds to. Clearer prompts generally get better results., chat modelchat model, n.: A language model trained to hold a back-and-forth conversation, keeping track of who said what and replying as a helpful assistant. It is the engine inside chat apps., evaluation harnessevaluation harness, n.: Software that puts AI models through tests in a fixed, repeatable way — feeding in the questions, collecting the answers and scoring them — so that every model is judged identically., instruction datasetinstruction dataset, n.: A collection of example requests paired with good answers, used to teach a pretrained language model to follow instructions and act as a helpful assistant., mid-trainingmid-training, n.: A middle stage in building a language model, between its broad initial training and its tuning into an assistant: it keeps learning the same way, but from a smaller, carefully chosen diet of math, code, reasoning or long documents., OllamaOllama, prop. n.: A free app that makes it easy to download and run AI language models on your own computer with a single command, keeping your data on your machine., prompt templateprompt template, n.: A reusable prompt with blanks to fill in — for example, 'Summarize this email for a busy manager: {email}' — so a program can send the same tested instructions with different content each time., special tokenspecial token, n.: A reserved symbol in an AI model's vocabulary that is not ordinary text but a signal — 'start of message', 'end of answer', 'begin thinking' — used to structure what the model reads and writes., supervised fine-tuningsupervised fine-tuning, n.: Teaching a pretrained AI model by showing it many examples of good answers to requests, so it learns to imitate them — like learning a job by studying worked examples., system promptsystem prompt, n.: Background instructions given to a chatbot before the conversation starts, setting its role, rules, tone and tools. Users usually never see it, but it shapes every reply., tokenizertokenizer, n.: The component that splits text into tokens and converts them to numbers before an AI model sees it, then turns the model's output numbers back into text., GGUFGGUF, abbr.: A file format that packs an AI language model, often in a shrunken low-precision version, into one file that ordinary computers can load and run quickly; used by llama.cpp and Ollama.