Skip to content
Last Dictionary
Plane
Generative MediaVision, Speech & Perception

inpainting

nounAlso: image inpainting, inpaint, outpainting, generative fill, image completion

In plain wordsFilling in a missing or erased part of a picture so it blends seamlessly with the rest — removing a stranger from a photo, or repairing a damaged one.

  1. The reconstruction of missing, damaged or user-masked regions of an image or video so that the result is plausible and consistent with the surrounding content. Classical methods propagated surrounding structure inward by solving partial differential equations, or copied matching patches from elsewhere in the image (exemplar-based methods, PatchMatch). Deep-learning methods train networks to fill masked regions, first with reconstruction and adversarial losses and now mostly with diffusion modelsdiffusion model, n.: An AI that learns to create images, sound or video by practicing noise removal: shown pictures buried under static, it learns to clean them up, then builds new ones starting from pure static., which are either fine-tuned to receive the mask and masked image as extra inputs or, without retraining, repeatedly overwrite the known region with appropriately noised copies of the original during sampling. A text prompt can specify what should appear in the hole. Outpainting extends an image beyond its borders by the same means, and commercial tools market prompt-driven inpainting as 'generative fill'. Video inpainting removes objects consistently across frames.

    Mask the tourist, inpaint 'empty cobblestone street', and the photo looks untouched.

Origin

Borrowed from art conservation, where inpainting means retouching losses in a painting. Marcelo Bertalmío, Guillermo Sapiro, Vicent Caselles and Coloma Ballester introduced the term to image processing with 'Image Inpainting' at SIGGRAPH 2000. OpenAI popularized 'outpainting' when it added the feature to DALL·E 2 in 2022.

Related

See also
  • diffusion modeldiffusion model, n.: An AI that learns to create images, sound or video by practicing noise removal: shown pictures buried under static, it learns to clean them up, then builds new ones starting from pure static.
  • image-to-image translationimage-to-image translation, n.: Turning one picture into another version of itself — a sketch into a photo, a day scene into night, a horse into a zebra — while keeping its layout.
  • instruction-based image editinginstruction-based image editing, n.: Changing a picture by telling an AI what to change — 'make it night', 'remove the car', 'give her a red coat' — instead of using editing tools by hand.
  • super-resolutionsuper-resolution, n.: Making a small or blurry image larger and sharper by having AI fill in plausible fine detail, rather than just stretching the pixels.
  • text-to-image modeltext-to-image model, n.: An AI system that creates a picture from a written description, such as 'a fox reading a newspaper, in watercolor', usually within seconds.
  • deepfakedeepfake, n.: A fake but realistic video, photo or voice recording made with AI that puts words in a real person's mouth or shows them in a scene that never happened.

Sources

  1. Bertalmío, M., Sapiro, G., Caselles, V. & Ballester, C. (2000). Image Inpainting. Proceedings of SIGGRAPH 2000, 417–424.

Mentioned in

AI artAI art, n.: Images, music or other artworks made with the help of AI — from early computer drawing programs to today's prompt-based image generators — and the debate over whether and how such work counts as art., text-to-image modeltext-to-image model, n.: An AI system that creates a picture from a written description, such as 'a fox reading a newspaper, in watercolor', usually within seconds., DALL·EDALL·E, prop. n.: OpenAI's pioneering series of image generators, named after the painter Salvador Dalí and the robot WALL·E, which showed in 2021–2023 that AI could draw almost anything described in words., image-to-image translationimage-to-image translation, n.: Turning one picture into another version of itself — a sketch into a photo, a day scene into night, a horse into a zebra — while keeping its layout., instruction-based image editinginstruction-based image editing, n.: Changing a picture by telling an AI what to change — 'make it night', 'remove the car', 'give her a red coat' — instead of using editing tools by hand.