action chunking
In plain wordsHaving a robot's AI plan a short burst of upcoming movements, say the next second, all at once instead of one tiny motion at a time, which makes motion smoother and mistakes rarer.
The practice of having a robot policypolicy, n.: An AI agent's strategy: the rule it follows to pick an action in each situation. Training in reinforcement learning means improving this rule until it earns high rewards. predict a sequence of k future actions (an action chunk) from each observation, rather than only the next action, and executing all or part of it before querying the policy again. Because the policy is consulted only once per chunk, a task of T steps needs about T/k decisions instead of T, giving small errors fewer chances to compound as they do in behavioral cloningbehavioral cloning, n.: The simplest way to copy an expert: record what the expert did in each situation and train an AI to predict the same action whenever it sees a similar situation.; a chunk can also carry timing, such as the pauses in human demonstrations, that the current observation alone does not explain. Overlapping chunks can be blended: ACT's temporal ensembling queries the policy every step and averages the predictions made for each timestep with exponentially decaying weights. Chunks of tens of steps, such as 50 actions covering one second of 50 Hz control, are typical in diffusiondiffusion policy, n.: A way of training robots in which the same kind of AI that generates images instead generates the robot's next movements, letting it copy demonstrations even when there are several good ways to do a task. and flow-matchingflow matching, n.: A way to train generators that learns, for every point between random noise and real data, which direction to move, so that noise can be steered smoothly into new examples. policies and in vision-language-action modelsvision-language-action model, n.: An AI model that looks through a robot's cameras, reads an instruction such as 'put the cup in the sink', and turns both straight into the robot's movements.. The cost is slower reaction to surprises within a chunk, which is mitigated by shorter execution windows or by blending each new chunk into the one still running.
With action chunking at k = 50 the gripper stopped hesitating mid-grasp.
Origin
Named in Tony Zhao, Vikash Kumar, Sergey Levine and Chelsea Finn's April 2023 paper 'Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware', which introduced Action Chunking with Transformers (ACT) and the ALOHA teleoperation rig, built for under US$20,000. The authors borrowed 'chunking' from psychology and neuroscience, where it describes grouping actions into units executed as one.
Usage note
Unrelated to chunkingchunking, n.: Cutting long documents into smaller pieces before storing them for AI search, so the system can find and hand the model just the relevant part instead of a whole manual. in retrieval-augmented generation, which splits documents into passages, and to chunking in cognitive architectures such as Soar.
Related
- diffusion policydiffusion policy, n.: A way of training robots in which the same kind of AI that generates images instead generates the robot's next movements, letting it copy demonstrations even when there are several good ways to do a task.
- behavioral cloningbehavioral cloning, n.: The simplest way to copy an expert: record what the expert did in each situation and train an AI to predict the same action whenever it sees a similar situation.
- imitation learningimitation learning, n.: Teaching an AI by showing it examples of an expert doing a task, so it learns to copy the behavior instead of discovering it through trial and error.
- vision-language-action modelvision-language-action model, n.: An AI model that looks through a robot's cameras, reads an instruction such as 'put the cup in the sink', and turns both straight into the robot's movements.
- teleoperationteleoperation, n.: Controlling a robot or machine from a distance, with a person steering it in real time using a joystick, a matching 'puppet' arm, a VR headset or a screen.
- transformertransformer, n.: The design behind today's chatbots and much of modern AI. It handles text by letting every word look at every other word and decide which of them matter for understanding it.
Sources
- Zhao, T. Z., Kumar, V., Levine, S. & Finn, C. (2023). Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware. Robotics: Science and Systems. arXiv:2304.13705. arxiv.org/abs/2304.13705
Mentioned in
actionaction, n.: A choice an AI agent makes at one moment — move left, place a stone, type a command — which changes its situation and may earn it a reward., action spaceaction space, n.: The full menu of moves an AI agent is allowed to make — a handful of buttons in a video game, or any combination of motor forces for a robot arm., behavioral cloningbehavioral cloning, n.: The simplest way to copy an expert: record what the expert did in each situation and train an AI to predict the same action whenever it sees a similar situation., robot learningrobot learning, n.: Teaching robots skills from data and experience, such as watching demonstrations, practicing by trial and error or training in simulation, instead of programming every movement by hand., teleoperationteleoperation, n.: Controlling a robot or machine from a distance, with a person steering it in real time using a joystick, a matching 'puppet' arm, a VR headset or a screen., vision-language-action modelvision-language-action model, n.: An AI model that looks through a robot's cameras, reads an instruction such as 'put the cup in the sink', and turns both straight into the robot's movements., Chelsea FinnChelsea Finn, prop. n.: An American computer scientist at Stanford known for 'learning to learn' methods that let AI adapt to new tasks from a few examples, and a co-founder of the robotics company Physical Intelligence., diffusion policydiffusion policy, n.: A way of training robots in which the same kind of AI that generates images instead generates the robot's next movements, letting it copy demonstrations even when there are several good ways to do a task., π0π0, prop. n.: A general-purpose AI model for controlling robots, released in 2024 by the start-up Physical Intelligence, that sees through cameras, follows typed or spoken instructions and moves robot arms deftly enough to fold laundry., Sergey LevineSergey Levine, prop. n.: A Berkeley computer scientist, one of the most prolific researchers on how robots can learn skills from data and experience, and a co-founder of the robotics company Physical Intelligence.