Architectures for Stereo Vision - Signal Processing Systems - page 475

a b

Fig. 2 Epipolar geometry: in ( a ) an unrectified setup and in ( b ) a rectified setup is shown. The
rectification process in ( b ) to achieve row-aligned search space is illustrated only for the left
projection plane

The rectified stereo setup is shown in Fig. 2 b and the disparity is purely horizontal
offset d

=(

x l −

)[ ] [ ] [ ]

x r pixel . With the rectified focal length f pixel , the baseline T m

[ ]

of the camera pair, the distance z m between the baseline and the 3D point can be

calculated as

fT fT

d .

z =

x r ) =

(1)

( x l −

This is also referred to as standard rectified geometry [ 83 ] . Thus, extracting depth
information from a stereo camera setup becomes estimating the disparity map
d .

In addition to a non-ideal camera setup, stereo vision systems have to handle
camera-inflicted image distortions, of which the most common are radial lens
distortion, sensor tilting and offset from focal axis [ 11 ] . These must be compensated
before rectification. However, when applying undistortion and rectification to a
sequence of input images both steps can be combined. Reverse mapping assigns
every pixel in the undistorted and rectified image a sub-pixel accurate origin in the
input image. The rectified pixels are obtained using any desired pixel interpolation
method. The bilinear interpolation for example, exhibits a reasonable trade-off
between image quality and hardware implementation costs. Alternative interpola-
tion methods are spline interpolation, which has higher silicon area requirements,
and nearest-neighbor, which does not provide the required resolution for disparity
estimation. Intermediate results from the processing steps are shown in Fig. 3 .
The displacement vectors for undistortion and rectification are calculated using
the intrinsic and extrinsic matrices, the tangential and the radial distortion parame-
ters. These can be obtained by separate camera calibration steps (e.g. [ 104 ] ) using
a calibration pattern, such as a chessboard pattern employed in OpenCV [ 11 ] .
Alternatively, or additionally, camera self-calibration from scene structure can be
employed for particular camera parameters. For latter use in e.g. cars, camera self-
calibration or at least updating of the intrinsic parameters from scene structure is
mandatory.

( x , y )