Skip to content
Last Dictionary
Plane
Generative MediaVision, Speech & Perception

super-resolution

nounAlso: SR, image super-resolution, single-image super-resolution, upscaling, AI upscaling, SRGAN

In plain wordsMaking a small or blurry image larger and sharper by having AI fill in plausible fine detail, rather than just stretching the pixels.

  1. 1

    Machine learning

    The task of producing a high-resolution image, video or audio signal from a lower-resolution input. Single-image super-resolution is ill-posed — many sharp images share the same low-resolution version — so learned models supply plausible detail from the statistics of their training data. Early networks such as SRCNN (2014) minimized pixel error and produced over-smooth results; SRGAN (2016) added adversarial and perceptual lossesperceptual loss, n.: A way of judging how different two images look to a person by comparing what a trained image-recognition network 'sees' in each, instead of comparing pixels one by one. for realistic texture; and diffusion upscalers such as SR3 (2021) iteratively refine noise conditioned on the low-resolution input. Super-resolution stages form the cascades of models such as ImagenImagen, prop. n.: Google's family of AI image generators, first shown in 2022, whose research showed that better understanding of language yields pictures that match descriptions more faithfully., power 'upscale' features in image tools, and run in real time in video games. Because the added detail is invented, outputs can contain hallucinatedhallucination, n.: When an AI states something false or made up — a fake fact, quotation or source — fluently and confidently, as if it were true. features, so learned upscaling is unsuitable as evidence or for medical diagnosis without great care.

    Run the 4× super-resolution pass last, after the face fixes.

  2. 2

    Imaging

    In optics, microscopy and remote sensing, techniques that recover detail beyond a system's nominal resolution limit — for example by fusing several frames shifted by fractions of a pixel, or, in super-resolution fluorescence microscopy, by localizing individual fluorescent molecules more precisely than the diffraction limit would otherwise allow.

Origin

The term is older than deep learning: multi-frame image super-resolution was studied in signal processing from the 1980s, and super-resolved fluorescence microscopy earned the 2014 Nobel Prize in Chemistry. Chao Dong and colleagues' SRCNN (ECCV 2014) opened the deep-learning era of the task.

Related

See also
  • image-to-image translationimage-to-image translation, n.: Turning one picture into another version of itself — a sketch into a photo, a day scene into night, a horse into a zebra — while keeping its layout.
  • perceptual lossperceptual loss, n.: A way of judging how different two images look to a person by comparing what a trained image-recognition network 'sees' in each, instead of comparing pixels one by one.
  • diffusion modeldiffusion model, n.: An AI that learns to create images, sound or video by practicing noise removal: shown pictures buried under static, it learns to clean them up, then builds new ones starting from pure static.
  • generative adversarial networkgenerative adversarial network, n.: Two neural networks trained as rivals: one forges fake images, the other tries to spot the fakes. As both improve, the forger's output becomes realistic enough to fool people.
  • hallucinationhallucination, n.: When an AI states something false or made up — a fake fact, quotation or source — fluently and confidently, as if it were true.
  • ImagenImagen, prop. n.: Google's family of AI image generators, first shown in 2022, whose research showed that better understanding of language yields pictures that match descriptions more faithfully.

Sources

  1. Dong, C., Loy, C. C., He, K. & Tang, X. (2016). Image Super-Resolution Using Deep Convolutional Networks. IEEE TPAMI. arXiv:1501.00092. arxiv.org/abs/1501.00092
  2. Ledig, C. et al. (2017). Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. CVPR 2017. arXiv:1609.04802. arxiv.org/abs/1609.04802
  3. Saharia, C. et al. (2021). Image Super-Resolution via Iterative Refinement. arXiv:2104.07636. arxiv.org/abs/2104.07636

Mentioned in

generative adversarial networkgenerative adversarial network, n.: Two neural networks trained as rivals: one forges fake images, the other tries to spot the fakes. As both improve, the forger's output becomes realistic enough to fool people., ImagenImagen, prop. n.: Google's family of AI image generators, first shown in 2022, whose research showed that better understanding of language yields pictures that match descriptions more faithfully., image-to-image translationimage-to-image translation, n.: Turning one picture into another version of itself — a sketch into a photo, a day scene into night, a horse into a zebra — while keeping its layout., inpaintinginpainting, n.: Filling in a missing or erased part of a picture so it blends seamlessly with the rest — removing a stranger from a photo, or repairing a damaged one., implicit neural representationimplicit neural representation, n.: A way of storing an image, shape or scene as a small neural network that, given any coordinate, returns what is there, such as a color or density, instead of as a grid of pixels., perceptual lossperceptual loss, n.: A way of judging how different two images look to a person by comparing what a trained image-recognition network 'sees' in each, instead of comparing pixels one by one., transposed convolutiontransposed convolution, n.: An operation that roughly reverses an image network's shrinking steps, enlarging a small grid of features into a bigger one, as when a model paints a full-size image.