Toolformer
In plain wordsA 2023 Meta research model that learned on its own when a tool — a calendar, a translator, a search engine, a calculator — would help it predict text, keeping only the tool uses that actually paid off.
A language modellanguage model, n.: A system that has learned how likely different sequences of words are, so it can predict the next word in a text and, by doing that repeatedly, write text of its own. from Meta AI researchers (February 2023) that taught itself, in a self-supervisedself-supervised learning, n.: A way for AI to learn from huge amounts of unlabeled data by setting itself fill-in-the-blank puzzles, such as guessing a hidden word or a missing patch of an image. way, to use tools by writing API calls into ordinary text. Shown only a few examples per tool, the base model, GPT-J, proposed calls throughout a large corpus — to a calendar, a machine-translation system, a question-answering system, a calculator and two search engines; the calls were executed, and a call was kept only if its result made the following tokens easier to predict (lower loss). Fine-tuning on this self-labeled data yielded a model that chooses for itself whether to call a tool, which one, with what input and how to use the answer. The authors reported zero-shot gains on several benchmarks, at times matching far larger models, with no loss of ordinary language-modeling ability.
Toolformer's loss-based filter is a neat way to get tool-use training data without human labels.
Origin
Described by Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda and Thomas Scialom in a paper posted on 9 February 2023 and published at NeurIPS 2023; the name blends 'tool' and 'transformertransformer, n.: The design behind today's chatbots and much of modern AI. It handles text by letting every word look at every other word and decide which of them matter for understanding it.'.
Usage note
Toolformer made one call at a time and could not chain tools or use them interactively; later systems rely on function callingfunction calling, n.: A feature of AI model services that lets the model reply with a neatly formatted request to run one of a program's functions, rather than with ordinary text; the program, not the model, then runs it. formats and on reinforcement learningreinforcement learning, n.: A way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time. in agent environments.
Related
- tool usetool use, n.: An AI model's ability to reach beyond its own knowledge by operating outside tools — searching the web, running code, using a calculator or an app — and folding the results into its work.
- function callingfunction calling, n.: A feature of AI model services that lets the model reply with a neatly formatted request to run one of a program's functions, rather than with ordinary text; the program, not the model, then runs it.
- augmented language modelaugmented language model, n.: A language model hooked up to extras — a search engine, a calculator, a memory store or other tools — so it can do things a model working only from its training cannot.
- self-supervised learningself-supervised learning, n.: A way for AI to learn from huge amounts of unlabeled data by setting itself fill-in-the-blank puzzles, such as guessing a hidden word or a missing patch of an image.
- ReActReAct, prop. n.: A method in which an AI alternates between thinking out loud about what to do next and taking an action, such as a web search, then uses what it observed to plan its next step.
- MRKL systemMRKL system, n.: An early design, pronounced 'miracle', in which a language model acts as a switchboard, sending each part of a question to the right specialist — a calculator, a database, a search tool — and combining the answers.
Sources
- Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Zettlemoyer, L., Cancedda, N. & Scialom, T. (2023). Toolformer: Language Models Can Teach Themselves to Use Tools. arXiv:2302.04761 (NeurIPS 2023). arxiv.org/abs/2302.04761v1
Mentioned in
tooltool, n.: Any outside capability an AI model is allowed to use — a web search, a calculator, a code runner, a database or an app — described to the model so it can ask for it by name., tool usetool use, n.: An AI model's ability to reach beyond its own knowledge by operating outside tools — searching the web, running code, using a calculator or an app — and folding the results into its work., augmented language modelaugmented language model, n.: A language model hooked up to extras — a search engine, a calculator, a memory store or other tools — so it can do things a model working only from its training cannot., MRKL systemMRKL system, n.: An early design, pronounced 'miracle', in which a language model acts as a switchboard, sending each part of a question to the right specialist — a calculator, a database, a search tool — and combining the answers.