score distillation sampling

A technique for fitting a differentiable generator — most often a 3D scene such as a neural radiance fieldneural radiance field, n.: A way of capturing a real scene from a set of photos as a small neural network that can then show the scene from any new viewpoint, as if moving a camera through it. or Gaussian splatsGaussian splatting, n.: A way of turning photos of a real place into a 3D scene made of millions of tiny, soft, colored blobs that can be viewed from any angle in real time, even on a phone or headset. — so that its renderings look plausible to a frozen, pretrained 2D diffusion modeldiffusion model, n.: An AI that learns to create images, sound or video by practicing noise removal: shown pictures buried under static, it learns to clean them up, then builds new ones starting from pure static. acting as the prior. At each iteration the scene is rendered from a random camera view, the rendering is noised to a random level, and the frozen diffusion model's predicted noise (under the text prompt, with strong classifier-free guidanceclassifier-free guidance, n.: A setting that makes an image generator follow the prompt more strictly by comparing what it would draw with and without the prompt, then exaggerating the difference.) is compared with the noise actually added; the difference, back-propagated through the renderer while skipping the diffusion network's own Jacobian, nudges the scene toward renderings the model finds likely. No 3D examples are needed: text-to-3D generation becomes possible with a 2D image model alone. SDS tends to give oversaturated, over-smoothed results and inconsistent multi-faced 'Janus' objects, which variants such as variational score distillation address; related gradients are used in diffusion distillationdiffusion distillation, n.: Training a fast image generator to imitate a slow diffusion model, so that a picture that once took fifty steps can be made in one to four.. Feed-forward 3D generators trained on 3D datasets are now common alternatives.