Skip to content
Last Dictionary
Plane
Vision, Speech & PerceptionRobotics & Embodied AI

structure from motion

nounAlso: SfM, bundle adjustment, COLMAP, incremental SfM, global SfM

In plain wordsRecovering both the 3D shape of a scene and the spot from which each photo was taken, using nothing but a collection of overlapping photographs.

  1. The problem, and family of methods, of jointly estimating a static scene's 3D structure and the poses of the cameras that photographed it from a set of overlapping images, without knowing the camera positions in advance. A typical pipeline detects and matches local features (SIFTscale-invariant feature transform, n.: A classic method for finding distinctive spots in a photo, such as corners and blobs, and describing them so the same spots can be matched in other photos taken from different distances or angles. or learned matchers) across image pairs; estimates the relative geometry of each pair (the essential or fundamental matrix) inside the RANSAC algorithm to reject false matches; adds cameras one at a time (incremental SfM) or solves for all rotations and positions together (global SfM); triangulates 3D points; and refines everything by bundle adjustment, a nonlinear least-squares optimization that minimizes the reprojection error between observed and predicted image points. The output, a sparse point cloudpoint cloud, n.: A 3D picture made of many dots floating in space, each marking a spot on a surface, as captured by a laser scanner or worked out from photos. plus calibrated cameras, is the usual starting point for multi-view stereo, neural radiance fieldsneural radiance field, n.: A way of capturing a real scene from a set of photos as a small neural network that can then show the scene from any new viewpoint, as if moving a camera through it. and Gaussian splattingGaussian splatting, n.: A way of turning photos of a real place into a 3D scene made of millions of tiny, soft, colored blobs that can be viewed from any angle in real time, even on a phone or headset.. Widely used systems include Bundler (Photo Tourism, 2006) and COLMAP (2016); learned feed-forward models such as VGGT (2025) now predict cameras and structure directly.

    Run structure from motion on the frames first; the splatting step needs the camera poses.

min⁡{Ri,ti},{Xj}∑i,j∥xij−π ⁣(Ki[Ri∣ti] Xj)∥2\min_{\{\mathbf{R}_i,\mathbf{t}_i\},\{\mathbf{X}_j\}} \sum_{i,j} \left\lVert \mathbf{x}_{ij} - \pi\!\left(\mathbf{K}_i[\mathbf{R}_i \mid \mathbf{t}_i]\,\mathbf{X}_j\right) \right\rVert^2

Origin

The phrase is usually traced to Shimon Ullman's 'The Interpretation of Structure from Motion' (1979), which asked how humans and machines infer 3D shape from moving 2D projections, building on the psychologists' 'kinetic depth effect'. Large-scale reconstruction from photo collections followed Noah Snavely, Steven Seitz and Richard Szeliski's Photo Tourism (2006).

Usage note

In the formula (bundle adjustment), x_ij is the observed image position of 3D point X_j in camera i; K_i, R_i and t_i are that camera's intrinsics, rotation and translation; π is perspective projection. SfM assumes a static scene. Real-time sequential variants are simultaneous localization and mappingsimultaneous localization and mapping, n.: How a robot, drone or headset builds a map of an unfamiliar place while at the same time working out where it is on that map, using its cameras or laser scanners..

Related

See also
  • 3D reconstruction3D reconstruction, n.: Building a 3D model of an object or place, its shape and often its colors, from photos, video or depth-sensor scans.
  • scale-invariant feature transformscale-invariant feature transform, n.: A classic method for finding distinctive spots in a photo, such as corners and blobs, and describing them so the same spots can be matched in other photos taken from different distances or angles.
  • point cloudpoint cloud, n.: A 3D picture made of many dots floating in space, each marking a spot on a surface, as captured by a laser scanner or worked out from photos.
  • simultaneous localization and mappingsimultaneous localization and mapping, n.: How a robot, drone or headset builds a map of an unfamiliar place while at the same time working out where it is on that map, using its cameras or laser scanners.
  • neural radiance fieldneural radiance field, n.: A way of capturing a real scene from a set of photos as a small neural network that can then show the scene from any new viewpoint, as if moving a camera through it.
  • Gaussian splattingGaussian splatting, n.: A way of turning photos of a real place into a 3D scene made of millions of tiny, soft, colored blobs that can be viewed from any angle in real time, even on a phone or headset.
  • pose estimationpose estimation, n.: Working out the position and orientation of something from images: where a person's joints are and how their body is posed, or exactly how an object or camera is placed and turned in space.

Sources

  1. Ullman, S. (1979). The Interpretation of Structure from Motion. Proceedings of the Royal Society of London B 203, 405–426.
  2. Schönberger, J. L. & Frahm, J.-M. (2016). Structure-from-Motion Revisited. CVPR 2016.

Mentioned in

3D reconstruction3D reconstruction, n.: Building a 3D model of an object or place, its shape and often its colors, from photos, video or depth-sensor scans., depth estimationdepth estimation, n.: Working out how far away each point in a scene is from the camera, producing a 'depth map' that turns a flat picture into a sense of near and far., Gaussian splattingGaussian splatting, n.: A way of turning photos of a real place into a 3D scene made of millions of tiny, soft, colored blobs that can be viewed from any angle in real time, even on a phone or headset., keypoint detectionkeypoint detection, n.: Finding specific, recognizable points in an image, such as the corners of a building, a person's elbows and knees, or the tip of a nose., neural radiance fieldneural radiance field, n.: A way of capturing a real scene from a set of photos as a small neural network that can then show the scene from any new viewpoint, as if moving a camera through it., point cloudpoint cloud, n.: A 3D picture made of many dots floating in space, each marking a spot on a surface, as captured by a laser scanner or worked out from photos., pose estimationpose estimation, n.: Working out the position and orientation of something from images: where a person's joints are and how their body is posed, or exactly how an object or camera is placed and turned in space., scale-invariant feature transformscale-invariant feature transform, n.: A classic method for finding distinctive spots in a photo, such as corners and blobs, and describing them so the same spots can be matched in other photos taken from different distances or angles.