Graphics Reference
In-Depth Information
Euclidean reconstructionalgorithm. In real-time applications, the camera calibration
matrix is usually assumed to be known.
Lourakis and Argyros [
304
] discussed an approach for real-time matchmoving
similar to the methods mentioned in Section
6.5.1.2
. Instead of using fundamental
matrices relating successive image pairs or trifocal tensors relating successive triples,
they proposed to track a 3D plane through the video sequence. The plane can be
specified as a dominant plane in the image (e.g., a wall or floor), or it can be a vir-
tual plane created as the best fit to the set of feature locations detected in the first
image pair. The projective transformation relating the plane's image in the current
frame to its image in the first frame is estimated, corresponding to a choice of the
free parameters in a camera matrix of the form of Equation (
6.31
). The cameras'
known calibration matrices are then used to upgrade to a Euclidean reconstruction.
Mouragnonet al. [
340
] proposedanalgorithmfor real-timematchmoving that applies
a bundle adjustment only over the parameters of a few of the most recent cameras
chosen as keyframes, tomaintain computational tractability. Resectioning is applied
to obtain camera matrices corresponding to images between keyframes.
The most common application of real-time camera tracking is not in visual effects
but in robotics, where the problem is called
simultaneous location andmapping
,or
SLAM
. In this case, the camera is mounted to a mobile robot that tries to self-localize
with respect to its environment; the problemis also known as estimating
ego-motion
.
If the environment contains distinguishable landmarks with known 3D positions
(e.g., ARTags), then the camera's location can be found by resectioning. If the envi-
ronment is unknown, the robot simultaneously builds a map of its surroundings and
its location and heading with respect to the map.
The SLAM problem in robotics differs from the structure frommotion problem in
computer vision in several important ways:
•
The SLAM problem is almost always formulated in a probabilistic way. For
example, a SLAM algorithm typically maintains an estimate of the camera's
state (i.e., external parameters) as well as the uncertainty in this state (which
couldbe representedby a covariancematrix). This uncertainty estimate comes
from a probabilistic model of image formation and often incorporates a prior
motion model for the camera (e.g., smooth trajectories are more likely).
•
The SLAM problem often (but not always) assumes the camera moves along
a plane, reducing the number of degrees of freedom from six external
parameters to three (two translation parameters and a rotation angle).
•
The SLAM problem is often more focused on accurately reconstructing an
environment, which means that recognizing revisited features and closing
loops plays a bigger role.
•
Features for the SLAM problem often come from nonvisual (e.g., acoustic
or range) sensors rather than from images alone. The advantage is that 3D
positions of scene points can be directly measured by the robot, instead of
obtaining just the heading (direction) to a scene point given by a camera.
Davison et al. [
115
] described one example of a real-time SLAM algorithm in
which the camera can freely move through a 3D environment. They explicitly over-
parameterized a calibrated camera by its 3D position, orientation, velocity, and
angular velocity, and searched the environment for Shi-Tomasi features that are
