cross-embodiment learning
In plain wordsTraining one robot AI on data from many differently shaped robots, such as arms, robot dogs, drones and humanoids, so that skills learned on one body help another.
Training policiespolicy, n.: An AI agent's strategy: the rule it follows to pick an action in each situation. Training in reinforcement learning means improving this rule until it earns high rewards. or representations on data from multiple robot embodiments — bodies that differ in kinematics, degrees of freedomdegrees of freedom, n.: The number of independent ways something can move. A robot arm with six motorized joints has six; a free-floating object can slide three ways and turn three ways., sensors, action spacesaction space, n.: The full menu of moves an AI agent is allowed to make — a handful of buttons in a video game, or any combination of motor forces for a robot arm. and control rates — so that experience gathered on one robot improves performance on others, including robots unseen in training. Approaches include expressing all robots' actions in a shared space such as end-effector displacements, tokenizing heterogeneous observations and actions so one transformertransformer, n.: The design behind today's chatbots and much of modern AI. It handles text by letting every word look at every other word and decide which of them matter for understanding it. can consume them, embodiment-specific input and output heads around a shared trunk, and learning from human videos by treating the human hand as one more embodiment. Evidence of positive transfer came from the RT-X models trained on Open X-EmbodimentOpen X-Embodiment, prop. n.: A large shared collection of robot training data, pooled by dozens of research labs in 2023 and covering many kinds of robots, so that one AI model could learn from all of them.; CrossFormer (2024) controlled arms, wheeled robots, quadcopters and quadrupeds with one set of weights trained on 900,000 trajectories from 20 embodiments. Commercial systems describe similar capabilities as motion transfer between robot types. The main risk is negative transfer when bodies are too dissimilar.
Cross-embodiment learning let the humanoid reuse grasps first demonstrated on a cheap tabletop arm.
Origin
Used for learning from demonstrators with different bodies in Kevin Zakka and colleagues' XIRL (Cross-embodiment Inverse Reinforcement Learning, June 2021). Earlier multi-robot transfer work includes Coline Devin and colleagues' modular policies (2017) and the RoboNet dataset (2019). The term spread with Open X-Embodiment (2023) and Ria Doshi and colleagues' 'Scaling Cross-Embodied Learning' (2024).
Usage note
Distinct from sim-to-real transfersim-to-real transfer, n.: Training a robot's AI in a computer simulation, where practice is fast, cheap and safe, and then making it work on the real robot despite the differences between simulation and reality., which moves a policy from simulation to hardware for the same robot, and from transfer learningtransfer learning, n.: Reusing what a model learned on one task to get a head start on another — like a pianist picking up the organ faster than a beginner. It lets models learn new jobs from little data. in general.
Related
- Open X-EmbodimentOpen X-Embodiment, prop. n.: A large shared collection of robot training data, pooled by dozens of research labs in 2023 and covering many kinds of robots, so that one AI model could learn from all of them.
- generalist robot policygeneralist robot policy, n.: A single robot-control program trained to do many different tasks in many places, often on several kinds of robots: the opposite of a specialist trained for one job.
- robot foundation modelrobot foundation model, n.: One large AI model trained on huge amounts of robot and internet data so that it can control many robots on many tasks, and be adapted quickly to new ones.
- transfer learningtransfer learning, n.: Reusing what a model learned on one task to get a head start on another — like a pianist picking up the organ faster than a beginner. It lets models learn new jobs from little data.
- vision-language-action modelvision-language-action model, n.: An AI model that looks through a robot's cameras, reads an instruction such as 'put the cup in the sink', and turns both straight into the robot's movements.
- degrees of freedomdegrees of freedom, n.: The number of independent ways something can move. A robot arm with six motorized joints has six; a free-floating object can slide three ways and turn three ways.
- sim-to-real transfersim-to-real transfer, n.: Training a robot's AI in a computer simulation, where practice is fast, cheap and safe, and then making it work on the real robot despite the differences between simulation and reality.
Sources
- Zakka, K. et al. (2021). XIRL: Cross-embodiment Inverse Reinforcement Learning. CoRL 2021. arXiv:2106.03911. arxiv.org/abs/2106.03911
- Doshi, R., Walke, H., Mees, O., Dasari, S. & Levine, S. (2024). Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviation. arXiv:2408.11812. arxiv.org/abs/2408.11812
Mentioned in
robot foundation modelrobot foundation model, n.: One large AI model trained on huge amounts of robot and internet data so that it can control many robots on many tasks, and be adapted quickly to new ones., robot learningrobot learning, n.: Teaching robots skills from data and experience, such as watching demonstrations, practicing by trial and error or training in simulation, instead of programming every movement by hand., sim-to-real transfersim-to-real transfer, n.: Training a robot's AI in a computer simulation, where practice is fast, cheap and safe, and then making it work on the real robot despite the differences between simulation and reality., vision-language-action modelvision-language-action model, n.: An AI model that looks through a robot's cameras, reads an instruction such as 'put the cup in the sink', and turns both straight into the robot's movements., generalist robot policygeneralist robot policy, n.: A single robot-control program trained to do many different tasks in many places, often on several kinds of robots: the opposite of a specialist trained for one job., Open X-EmbodimentOpen X-Embodiment, prop. n.: A large shared collection of robot training data, pooled by dozens of research labs in 2023 and covering many kinds of robots, so that one AI model could learn from all of them., Physical IntelligencePhysical Intelligence, prop. n.: A San Francisco company, founded in 2024, that builds general-purpose AI 'brains' meant to control many different kinds of robots doing many different tasks.