text-to-video model
In plain wordsAn AI that creates a short video clip — increasingly with matching sound — from a written description, such as 'a paper boat sailing down a rainy street'.
A generative modelgenerative model, n.: A model that learns what a kind of data looks like — faces, sentences, songs — well enough to produce new examples that resemble the originals without copying them. that produces video conditioned on a text prompt and, usually, on images (image-to-video) or other video. Most current systems are latent diffusionlatent diffusion model, n.: A diffusion image generator that works on a shrunken, compressed version of each picture instead of the full-size pixels, which makes it far cheaper to train and run. or flow matchingflow matching, n.: A way to train generators that learns, for every point between random noise and real data, which direction to move, so that noise can be steered smoothly into new examples. models: a video autoencoder compresses frames in space and time, and a diffusion transformerdiffusion transformer, n.: A diffusion image or video generator whose core network is a transformer — the architecture behind chatbots — instead of the older image-specific design, which lets it improve more predictably as it grows. denoises 'spacetime patches' of the latent, attending across frames so that objects, lighting and motion stay consistent; some models instead generate frames or tokens autoregressively. Training uses very large collections of captioned video, often recaptioned by vision-language models, together with images. Since 2025 leading models also generate synchronized audio — speech, sound effects and music. Clips typically last 5–30 seconds at up to 1080p or 4K, with tools for extension, camera control and character consistency; physics errors, lapses in object permanence and drift over long durations remain common. Prominent systems as of October 2026 include Google's VeoVeo, prop. n.: Google's video-generating AI, which since 2025 creates short clips complete with matching sound and dialogue from a written prompt. and Gemini Omni, ByteDance's Seedance, Kuaishou's Kling, Runway's Gen series and Alibaba's Wan, several versions of which have open weights; OpenAI discontinued SoraSora, prop. n.: OpenAI's video-generating AI, unveiled in 2024 with startlingly realistic clips; a 2025 version added sound and a social app, but OpenAI shut the service down in 2026. in 2026. Some researchers see such models as nascent world modelsworld model, n.: An AI's internal simulation of how its world works, which lets it predict what will happen next — especially in response to its own actions — and so plan, imagine or practice without acting for real.; others doubt that they learn physics. Their realism sharpens concerns about deepfakesdeepfake, n.: A fake but realistic video, photo or voice recording made with AI that puts words in a real person's mouth or shows them in a scene that never happened. and AI slopAI slop, n.: Low-quality content churned out in bulk by AI — bland articles, fake-looking images, filler videos — posted to grab clicks or fill space, with little care for accuracy or value..
The text-to-video model kept the cat consistent for eight seconds, then gave it a second tail.
Origin
Research prototypes generated tiny clips from about 2017. In 2022 CogVideo, Meta's Make-A-Video and Google's Imagen Video and Phenaki produced short clips from text; Runway's Gen-2 (2023) brought text-to-video to the public, OpenAI's Sora preview (February 2024) showed minute-long, largely coherent video, and Google's Veo 3 (May 2025) popularized synchronized audio.
Related
- SoraSora, prop. n.: OpenAI's video-generating AI, unveiled in 2024 with startlingly realistic clips; a 2025 version added sound and a social app, but OpenAI shut the service down in 2026.
- VeoVeo, prop. n.: Google's video-generating AI, which since 2025 creates short clips complete with matching sound and dialogue from a written prompt.
- diffusion transformerdiffusion transformer, n.: A diffusion image or video generator whose core network is a transformer — the architecture behind chatbots — instead of the older image-specific design, which lets it improve more predictably as it grows.
- latent diffusion modellatent diffusion model, n.: A diffusion image generator that works on a shrunken, compressed version of each picture instead of the full-size pixels, which makes it far cheaper to train and run.
- world modelworld model, n.: An AI's internal simulation of how its world works, which lets it predict what will happen next — especially in response to its own actions — and so plan, imagine or practice without acting for real.
- deepfakedeepfake, n.: A fake but realistic video, photo or voice recording made with AI that puts words in a real person's mouth or shows them in a scene that never happened.
- text-to-image modeltext-to-image model, n.: An AI system that creates a picture from a written description, such as 'a fox reading a newspaper, in watercolor', usually within seconds.
- AI slopAI slop, n.: Low-quality content churned out in bulk by AI — bland articles, fake-looking images, filler videos — posted to grab clicks or fill space, with little care for accuracy or value.
Sources
- Brooks, T., Peebles, B. et al. (2024). Video generation models as world simulators. OpenAI technical report. openai.com/index/video-generation-models-as-world-simulators
Mentioned in
promptprompt, n.: The message or instructions you give an AI system — a question, a request, some examples or a document — that it responds to. Clearer prompts generally get better results., text-to-image modeltext-to-image model, n.: An AI system that creates a picture from a written description, such as 'a fox reading a newspaper, in watercolor', usually within seconds., world modelworld model, n.: An AI's internal simulation of how its world works, which lets it predict what will happen next — especially in response to its own actions — and so plan, imagine or practice without acting for real., ByteDanceByteDance, prop. n.: The Chinese company that owns TikTok, which also runs Doubao, China's most popular AI chatbot, and builds image and video generators., SoraSora, prop. n.: OpenAI's video-generating AI, unveiled in 2024 with startlingly realistic clips; a 2025 version added sound and a social app, but OpenAI shut the service down in 2026., VeoVeo, prop. n.: Google's video-generating AI, which since 2025 creates short clips complete with matching sound and dialogue from a written prompt., CLIP scoreCLIP score, n.: A score for how well a picture matches a text description, computed by an AI model that was trained to pair images with their captions., MiniMax (company)MiniMax (company), prop. n.: A Shanghai artificial intelligence company, founded in 2021, that makes AI companion apps, the Hailuo video generator and large language models, several released openly., RunwayRunway, prop. n.: A New York company that makes AI tools for generating and editing video, used by filmmakers and artists; it helped create the image generator Stable Diffusion., Will Smith eating spaghettiWill Smith eating spaghetti, n.: A grotesque 2023 AI-generated video of actor Will Smith eating spaghetti that became a meme and an informal yardstick: each new video model is judged by how well it redoes the scene., world foundation modelworld foundation model, n.: A large AI model trained on vast amounts of video that can imagine how a scene will unfold, used to create realistic practice worlds for robots and self-driving cars.