generalist robot policy
In plain wordsA single robot-control program trained to do many different tasks in many places, often on several kinds of robots: the opposite of a specialist trained for one job.
A single learned policypolicy, n.: An AI agent's strategy: the rule it follows to pick an action in each situation. Training in reinforcement learning means improving this rule until it earns high rewards. that performs a wide range of tasks across varied scenes, objects and often several robot bodies, conditioned on a goal given as a language instruction, goal image or task label. It contrasts with the specialist policies that long dominated robot learningrobot learning, n.: Teaching robots skills from data and experience, such as watching demonstrations, practicing by trial and error or training in simulation, instead of programming every movement by hand. — one network per task, robot and environment — and is usually pretrained on large multi-task, multi-robot datasets, then fine-tuned with a small amount of in-domain data. Examples include RT-1, Octo (an open transformer policy trained on about 800,000 Open X-EmbodimentOpen X-Embodiment, prop. n.: A large shared collection of robot training data, pooled by dozens of research labs in 2023 and covering many kinds of robots, so that one AI model could learn from all of them. trajectories), CrossFormer and the vision-language-action modelsvision-language-action model, n.: An AI model that looks through a robot's cameras, reads an instruction such as 'put the cup in the sink', and turns both straight into the robot's movements.. Generalist policies are judged by success on unseen tasks and environments, by how efficiently they adapt, and by whether training on many tasks helps each one (positive transfer) rather than hurting it.
The generalist policy is worse than our specialist on bin picking but works on forty other tasks.
Origin
Echoes DeepMind's Gato, presented as 'a generalist agent' (Scott Reed and colleagues, May 2022), which also stacked blocks with a real robot arm. Earlier multi-task robot work was also called 'generalist', and the first use of the exact phrase is not established; it was popularized by 'Octo: An Open-Source Generalist Robot Policy' (Octo Model Team, May 2024) and then became common in paper titles and benchmarks.
Usage note
Overlaps with robot foundation modelrobot foundation model, n.: One large AI model trained on huge amounts of robot and internet data so that it can control many robots on many tasks, and be adapted quickly to new ones.: a generalist policy outputs actions directly, whereas a foundation model may instead plan, simulate or perceive for a robot. 'Generalist' is relative; current systems generalize within limited families of tasks.
Related
- robot foundation modelrobot foundation model, n.: One large AI model trained on huge amounts of robot and internet data so that it can control many robots on many tasks, and be adapted quickly to new ones.
- vision-language-action modelvision-language-action model, n.: An AI model that looks through a robot's cameras, reads an instruction such as 'put the cup in the sink', and turns both straight into the robot's movements.
- Open X-EmbodimentOpen X-Embodiment, prop. n.: A large shared collection of robot training data, pooled by dozens of research labs in 2023 and covering many kinds of robots, so that one AI model could learn from all of them.
- cross-embodiment learningcross-embodiment learning, n.: Training one robot AI on data from many differently shaped robots, such as arms, robot dogs, drones and humanoids, so that skills learned on one body help another.
- multi-task learningmulti-task learning, n.: Training one model to do several related jobs at once, such as spotting objects and judging distance in the same photo, so that practice on each job can sharpen the others.
- policypolicy, n.: An AI agent's strategy: the rule it follows to pick an action in each situation. Training in reinforcement learning means improving this rule until it earns high rewards.
Sources
- Octo Model Team (2024). Octo: An Open-Source Generalist Robot Policy. arXiv:2405.12213. arxiv.org/abs/2405.12213
- Reed, S. et al. (2022). A Generalist Agent. Transactions on Machine Learning Research.
Mentioned in
robot foundation modelrobot foundation model, n.: One large AI model trained on huge amounts of robot and internet data so that it can control many robots on many tasks, and be adapted quickly to new ones., vision-language-action modelvision-language-action model, n.: An AI model that looks through a robot's cameras, reads an instruction such as 'put the cup in the sink', and turns both straight into the robot's movements., cross-embodiment learningcross-embodiment learning, n.: Training one robot AI on data from many differently shaped robots, such as arms, robot dogs, drones and humanoids, so that skills learned on one body help another., GatoGato, prop. n.: A 2022 DeepMind AI that used one set of learned settings for hundreds of jobs, from chatting and describing pictures to playing video games and moving a real robot arm — an early step toward general-purpose AI agents., Open X-EmbodimentOpen X-Embodiment, prop. n.: A large shared collection of robot training data, pooled by dozens of research labs in 2023 and covering many kinds of robots, so that one AI model could learn from all of them., Physical IntelligencePhysical Intelligence, prop. n.: A San Francisco company, founded in 2024, that builds general-purpose AI 'brains' meant to control many different kinds of robots doing many different tasks., π0π0, prop. n.: A general-purpose AI model for controlling robots, released in 2024 by the start-up Physical Intelligence, that sees through cameras, follows typed or spoken instructions and moves robot arms deftly enough to fold laundry.