world foundation model
In plain wordsA large AI model trained on vast amounts of video that can imagine how a scene will unfold, used to create realistic practice worlds for robots and self-driving cars.
A large generative model of how the physical world unfolds, pretrained on huge video collections and often on robot or driving sensor logs, that continues a scene — as future frames or latent states — from a prompt of text, images, earlier frames or actions; it is meant as a general starting point from which narrower world modelsworld model, n.: An AI's internal simulation of how its world works, which lets it predict what will happen next — especially in response to its own actions — and so plan, imagine or practice without acting for real. for a given robot, vehicle or setting are adapted by fine-tuning. Used as learned simulators, such models generate varied, physically plausible synthetic datasynthetic data, n.: Data created by computer programs or AI models rather than collected from the real world — such as practice math problems written by a model, or simulated street scenes for self-driving cars. (new weather, lighting or rare traffic events), restyle simulator renderings toward photorealism, evaluate policiespolicy, n.: An AI agent's strategy: the rule it follows to pick an action in each situation. Training in reinforcement learning means improving this rule until it earns high rewards. by imagining their consequences, and serve as backbones for planning or for vision-language-action modelsvision-language-action model, n.: An AI model that looks through a robot's cameras, reads an instruction such as 'put the cup in the sink', and turns both straight into the robot's movements.. Architectures include diffusion transformersdiffusion transformer, n.: A diffusion image or video generator whose core network is a transformer — the architecture behind chatbots — instead of the older image-specific design, which lets it improve more predictably as it grows. and autoregressive models over video tokens. Open questions concern physical fidelity — generated video can break object permanence or conservation laws — controllability by actions, compute cost, and how to measure physical understanding.
We used a world foundation model to turn one recorded drive into a hundred rainy-night variants.
Origin
Introduced by NVIDIA with its Cosmos platform, announced at CES in Las Vegas in January 2025 and described in 'Cosmos World Foundation Model Platform for Physical AI' (arXiv, 7 January 2025), which defined it as 'a general-purpose world model that can be fine-tuned into customized world models'; the models were released with open weights.
Usage note
Largely NVIDIA's term; others describe similar systems, such as Google DeepMind's interactive Genie models, simply as world models. See world modelworld model, n.: An AI's internal simulation of how its world works, which lets it predict what will happen next — especially in response to its own actions — and so plan, imagine or practice without acting for real. for the broader concept in reinforcement learning and cognitive science.
Related
- world modelworld model, n.: An AI's internal simulation of how its world works, which lets it predict what will happen next — especially in response to its own actions — and so plan, imagine or practice without acting for real.
- physical AIphysical AI, n.: AI built to work in the physical world rather than on a screen: the intelligence inside robots, self-driving cars, drones and other machines that must sense real surroundings and move through them.
- physics simulatorphysics simulator, n.: Software that imitates how real objects move, collide, bend and fall, so robots and their AI can be designed, trained and tested on a computer before touching the real world.
- synthetic datasynthetic data, n.: Data created by computer programs or AI models rather than collected from the real world — such as practice math problems written by a model, or simulated street scenes for self-driving cars.
- text-to-video modeltext-to-video model, n.: An AI that creates a short video clip — increasingly with matching sound — from a written description, such as 'a paper boat sailing down a rainy street'.
- robot foundation modelrobot foundation model, n.: One large AI model trained on huge amounts of robot and internet data so that it can control many robots on many tasks, and be adapted quickly to new ones.
- digital twindigital twin, n.: A detailed virtual copy of a real machine, factory, robot or city, kept in sync with live sensor data and used to monitor it, test changes safely and predict problems.
Sources
- NVIDIA et al. (2025). Cosmos World Foundation Model Platform for Physical AI. arXiv:2501.03575. arxiv.org/abs/2501.03575
- Robotics 24/7 (2025). CES 2025: NVIDIA launches Cosmos world foundation model, expands Omniverse. robotics247.com/article/ces_2025_nvidia_launches_cosmos_world_foundation_model_expands_omniverse
Mentioned in
autonomous vehicleautonomous vehicle, n.: A vehicle, usually a car, truck or shuttle, that can drive itself, sensing the road, deciding what to do and steering, braking and accelerating with no human driving., physical AIphysical AI, n.: AI built to work in the physical world rather than on a screen: the intelligence inside robots, self-driving cars, drones and other machines that must sense real surroundings and move through them., digital twindigital twin, n.: A detailed virtual copy of a real machine, factory, robot or city, kept in sync with live sensor data and used to monitor it, test changes safely and predict problems., physics simulatorphysics simulator, n.: Software that imitates how real objects move, collide, bend and fall, so robots and their AI can be designed, trained and tested on a computer before touching the real world., robot foundation modelrobot foundation model, n.: One large AI model trained on huge amounts of robot and internet data so that it can control many robots on many tasks, and be adapted quickly to new ones.