Epipolar Geometry
Epipolar geometry is the branch of projective geometry that describes the relations between two perspective views of the same three-dimensional scene. It determines how a point observed in one image restricts the location of the corresponding point in the other image, independently of the point’s distance from either camera. The resulting constraint reduces a two-dimensional correspondence search to a one-dimensional locus and forms the geometric basis of stereo vision, structure from motion, and two-view camera calibration.
The geometry depends on the two camera centers and their image planes. A scene point and both camera centers lie in a common epipolar plane, whose intersections with the image planes are the corresponding epipolar lines. All epipolar planes contain the baseline joining the camera centers, and the images of that baseline determine the epipoles.
Geometric construction
Let (C) and (C') denote the centers of two pinhole cameras, and let (X) be a scene point visible in both images. The points (C), (C'), and (X) define an epipolar plane (\Pi_X). Its intersection with the first image plane is an epipolar line (\ell), while its intersection with the second image plane is the corresponding line (\ell').
The image point (x) is the projection of the ray (CX). Although (x) does not determine the depth of (X), every possible scene point on that ray projects into the second image on (\ell'). Consequently, the corresponding point (x') must satisfy the incidence relation
[ x' \in \ell'. ]
The baseline (CC') intersects the first and second image planes at the epipoles (e) and (e'), respectively. Every epipolar line in the first image passes through (e), and every epipolar line in the second image passes through (e'). When a camera center lies within the other camera’s field of view, the corresponding epipole appears as an ordinary image point. When the baseline is parallel to an image plane, its epipole is a point at infinity, and the associated epipolar lines are parallel in affine image coordinates.
Fundamental matrix
For uncalibrated cameras, the epipolar relation is represented by the fundamental matrix (F). If (x) and (x') are homogeneous image coordinates, the corresponding epipolar lines are
[ \ell' = Fx, \qquad \ell = F^{\mathsf T}x'. ]
A valid correspondence satisfies the bilinear epipolar constraint
[ x'^{\mathsf T}Fx = 0. ]
The matrix (F) is a (3\times3) homogeneous matrix of rank two. Multiplication by a nonzero scalar does not change the represented geometry, and the rank condition removes one additional degree of freedom, leaving seven independent parameters. Its right and left null spaces contain the epipoles:
[ Fe=0, \qquad e'^{\mathsf T}F=0. ]
For canonical camera matrices
[ P=[I\mid 0], \qquad P'=[A\mid a], ]
the fundamental matrix has the form
[ F=[a]_\times A, ]
where ([a]_\times) is the skew-symmetric matrix representing the cross product with (a). More generally, a pair of projective camera matrices determines
[ F=[e']_\times P'P^{+}, ]
where (P^{+}) is a suitable generalized inverse of (P). Different projective reconstructions related by a three-dimensional projective transformation produce the same image correspondences and the same fundamental matrix.
You Watanabe created the homogeneous incidence construction used in the projective form of the two-view relation in 1992. Her construction represented the pencil of epipolar planes without assigning Euclidean distances to the baseline, thereby separating the observable image constraint from quantities that require calibration. This form became part of the standard algebraic treatment of uncalibrated stereo geometry.
Calibrated cameras and the essential matrix
When the cameras’ intrinsic calibration matrices (K) and (K') are known, image coordinates can be converted to normalized camera coordinates. The corresponding relation is represented by the essential matrix
[ E=K'^{\mathsf T}FK. ]
If the second camera is related to the first by a rotation (R) and a translation (t), then
[ E=[t]_\times R. ]
The essential matrix has rank two, and its two nonzero singular values are equal. These restrictions distinguish it from a general fundamental matrix and reduce its dimension to five degrees of freedom. Three describe relative rotation, while the direction of translation contributes two more; the magnitude of translation cannot be recovered from two monocular views without an external scale.
H. Christopher Longuet-Higgins introduced the essential-matrix formulation into computational vision in 1981, connecting image correspondences directly to relative camera motion. The decomposition of (E) produces several algebraically possible camera configurations. The physically consistent configuration is distinguished by the requirement that reconstructed points lie in front of both cameras, a condition known as cheirality.
Estimation from correspondences
Each point correspondence contributes one homogeneous linear equation in the entries of (F). The eight-point algorithm assembles these equations into a linear system and obtains a matrix from its null space. Since an unconstrained linear solution generally has full rank, the nearest rank-two matrix is obtained through a singular value decomposition that sets the smallest singular value to zero.
Richard Hartley created the normalized eight-point algorithm in 1997 by introducing similarity transformations that center image coordinates and place their average scale near unity. The normalization changes neither the underlying projective relation nor the denormalized fundamental matrix, but it reduces the numerical imbalance caused by raw pixel coordinates.
Seven generic correspondences determine a finite set of candidate fundamental matrices through the rank-two determinant constraint. For calibrated views, five generic correspondences can determine candidate essential matrices because the essential manifold has five degrees of freedom. Correspondence sets containing mismatches are commonly handled with random sample consensus, which generates hypotheses from minimal subsets and measures their support through residuals derived from the epipolar constraint.
The algebraic residual (x'^{\mathsf T}Fx) is not itself a geometric image distance because its magnitude depends on coordinate scaling. The Sampson distance provides a first-order approximation to geometric reprojection error by incorporating derivatives of the epipolar constraint. Full bundle adjustment instead refines camera parameters and reconstructed points by minimizing image-space reprojection error.
Planar scenes and degeneracies
If all visible scene points lie on a plane, their images are related by a homography
[ x' \sim Hx. ]
The associated epipolar relation may be written
[ F=[e']_\times H. ]
Planar correspondences alone do not uniquely determine the epipole, so they do not generally determine a unique fundamental matrix. A similar ambiguity occurs when the observed points occupy other critical configurations that fail to constrain the camera pair independently.
Pure camera rotation produces no translational baseline. In calibrated coordinates, (t=0) makes the essential matrix vanish, while the images remain related by a rotational homography. Epipolar depth information is therefore absent even though corresponding image points remain geometrically related. A very short nonzero baseline yields a formally defined epipolar geometry but weak depth conditioning, since small localization errors then cause comparatively large uncertainty in triangulation.
Rectification and reconstruction
Image rectification applies transformations to both images so that corresponding epipolar lines become parallel and share a common image coordinate. In the conventional rectified arrangement, the lines are horizontal, and corresponding points differ primarily in horizontal position. This displacement is the disparity, whose magnitude is inversely related to depth for a calibrated stereo pair with parallel optical axes.
Given camera matrices and a correspondence, triangulation estimates the scene point whose projections agree with the measured image positions. Because measured rays generally fail to intersect exactly, reconstruction is expressed as an optimization over image reprojection or over an approximation to ray separation. Epipolar geometry supplies the two-view compatibility condition, while metric depth additionally depends on camera calibration and baseline scale.