Graphics Reference
In-Depth Information
of a Euclidean reconstruction, all of the camera parameter and scene point estimates
are refined using a non-linear estimation step called
bundle adjustment
to make
the reconstruction match the image features as well as possible. Finally, we discuss
several practical issues for image sequence calibration, which becomes difficult for
very long sequences.
6.5.1
Projective Reconstruction
We now formally state the image sequence calibration problem. We detect and track
a set of
n
features across
m
images from a moving camera. Let the camera matrices
corresponding to these images be
, and the homogeneous coordinates of
the 3D scene points that generated each feature be
{
P
1
,
...
,
P
m
}
. The homogeneous
coordinates of the
j
th
feature in the image from the
i
th
camera are denoted
x
ij
. Since
every 3Dpoint may not be observed in each image, we also create a binary variable
{
X
1
,
...
,
X
n
}
χ
ij
that specifies whether the
j
th
point appears in the
i
th
image. That is, if
χ
ij
=
1, we have
x
ij
∼
P
i
X
j
(6.36)
Given the image projections
{
χ
ij
,
x
ij
}
as input, we want to determine the
unknown camera matrices
. Since we want
tofind cameramatrices and scene points that reproduce the projectionswe observed,
a natural approach is to minimize the sum of squared distances
{
P
1
,
...
,
P
m
}
and scene points
{
X
1
,
...
,
X
n
}
m
n
2
1
χ
ij
d
(
x
ij
,
P
i
X
j
)
(6.37)
i
=
1
j
=
where
d
is the Euclidean distance between two points on the image plane (i.e., after
we convert from homogeneous to unhomogeneous coordinates).
This minimization problem is generally called
bundle adjustment
, for the reason
illustrated in Figure
6.10
. That is, we are adjusting the “bundles” of rays emanating
fromeach camera to the scene points in order to bring the estimated projections onto
the image planes as close as possible to the observed feature locations. The quantity
in Equation (
6.37
) is also called the
reprojection error
.
We will discuss the numerical solution to this nonlinear problem in Section
6.5.3
.
However, thefirst consideration is determining a good initial estimateof theunknown
variables so that the bundle adjustment process starting from this initial guess con-
verges to a reasonable answer. We discuss twomethods: one based on a factorization
approach for all the cameras at once, and one built on sequentially estimating camera
matrices, exploiting the knowledge that the images come from a sequence.
6.5.1.1 Projective Factorization
Sturm and Triggs [
474
,
498
] proposed an elegant approach for projective reconstruc-
are visible in all the images (i.e.,
χ
=
1 for all
i
,
j
). Then we can collect the projection
ij
15
This approach was inspired by a classic algorithm by Tomasi and Kanade [
493
], who showed how
factorization applied to a simpler method of image formation, orthographic projection.
