Matchmoving - Computer Vision for Visual Effects - page 222

In this section, we describe the relationship between the fundamental matrix F
and the camera matrices P and P corresponding to a stereo rig. We assume that
the only information available for the estimation problems is a set of feature matches
between a single pair of images (one fromeach camera, taken simultaneously), which
can be obtained using any of the methods in Chapter 4 . We characterize the ambi-
guities mentioned previously, and describe how they can be resolved with additional
information.

6.4.1 Relating the Fundamental and Camera Matrices

When dealing with a stereo rig, we are predominantly interested in the relative
rotation matrix and translation vector relating the two cameras, since from point
correspondences alone there is no way to determine absolute position with respect
to the world coordinates. Therefore, we assume that the first camera is centered at
the origin of the world coordinate system and is aligned with the world coordinate
axes, and that the second camera matrix is expressed using a rotation and translation
with respect to the first camera's coordinate system:

P = K [

P = K [ I 3 × 3 | 0 3 × 1 ] R | t ] (6.27)

Now that we've defined the process of image formation, we can determine the
fundamental matrix F relating the two cameras in terms of the camera parameters
K , K , R , and t . Since the two camera centers are at

R t , respectively,
we can compute the epipoles by directly applying P and P to these points: 11

] and

[ 0, 0, 0 −

KR t e ∼ K t

e ∼ (6.28)

We can show (see Problem 6.10 ) that the fundamental matrix for the image pair is
given by

K t ] × K RK − 1

F =[ (6.29)

where we used the notation [·] × defined in Equation ( 5.39 ). This proves a claim we

made earlier in Equation ( 5.38 ), stating that F

e ] × M for some rank-3 matrix M .
When calibrating a stereo rig from feature matches, we first robustly estimate
the fundamental matrix (Section 5.4.2 ), and then extract consistent camera matrices
from F . 12 U nfortunately, this is where we run into the projective ambiguity referred
to earlier. That is, consider a projective transformation of the world coordinate sys-
tem given by a 4

=[

4 non-singular matrix H . If we consider a camera matrix P and
homogeneous world point X , from Equation ( 6.11 ) we have the projection x

×

∼ P X .

Now consider an alternate camera matrix P

X

H − 1 X ; since = PH and world point =

P X

PHH − 1 X = = P X ∼ x (6.30)

we get the same projected point on the image. Thus, from image correspondences
alone, we have no way to determine whether our estimates of the camera matrices

11 We removed the negative sign from the expression for e in Equation ( 6.28 ) since

∼ accounts for

this scalar multiple.
12 Zhang [ 574 ] and Fitzgibbon [ 144 ] describedmethods for simultaneously estimating lens distortion
coefficients and the fundamental matrix from a set of feature matches between an image pair.