Information Technology Reference
In-Depth Information
K
⋅
B
D
(
p
,
p
)
=
(3)
n
x
y
d
(
p
,
p
)
n
x
y
where
K
is the focal length of the video camera,
B
is the baseline distance between two
video cameras.
D
n
(
p
x
,
p
y
) is the real depth value corresponding to the measured dispar-
ity value
d
n
(
p
x
,
p
y
) at pixel position (
p
x
,
p
y
) in the image pattern depth image.
Thereafter, we generate a mapping curve between real depths and measured
depths from the depth camera and find the fitting curve using the cubic equation
y=a+bx+cx
2
+dx
3
. The cross small rectangular points on the
x
−
y
plane are formed
by the measured depths
x
and real depths
y
that minimizes the sum of squared dis-
tances to these points.
2.3 Depth Image Generation
Since the measuring distance of the depth camera is approximately from 1m to
4m, we cannot obtain depth information from far objects. In the video-plus-depth
generation, we regard the near objects captured by the depth camera as an ROI or
a foreground. First, we move the depth data captured by the depth camera to the
world coordinate, and then reproject the warped depth data onto the image plane
of the left camera.
When
D
s
(
p
sx
,
p
sy
) is the depth information at the pixel position (
p
sx
,
p
sy
) in the
depth image, we can regard the pixel
p
s
(
p
sx
,
p
sy
,
D
s
(
p
sx
,
p
sy
)) as a 3D point. The
corresponding point
p
l
of the left image is calculated by
−
1
p
=
P
'
⋅
P
⋅
p
(4)
l
l
s
s
where
P
l
′ and
P
s
-1
are the relative projection matrix of the left camera and the in-
verse relative projection matrix of the depth camera, respectively.
Figure 4 shows an initial ROI depth image that is overlapped onto the left image.
When we compare the original depth image with it, we can notice that the body region
is extended to fit with the spatially high-resolution left image. We can also notice that
holes occur in the initial ROI depth image due to the warping operation.
3D image warping
Original depth image
Initial ROI depth image
Fig. 4
Generation of initial ROI depth image
