Information Technology Reference
In-Depth Information
x
=
PX
,
(13)
where
x
denotes the 2D pixel coordinate and
X
denotes the 3D world coordinate,
respectively.
Once the 3D coordinates of the moving objects are determined, the frames at any
desired time instant could be interpolated by the help of the estimated rotation,
R
,
and translation,
t
, matrices.
At this point, the following crucial question arises: Will the depth maps and pro-
jection matrices be available for LCD panels in the homes of consumers? The an-
swer lies behind the fact that 3D display technologies have been progressed dras-
tically in the recent years. The glass-free auto-stereoscopic displays, which create
the perception of the 3
rd
dimension by presenting multiple views, are expected to
spread into the consumer market in the very near future. These new generation 3D
displays require texture and depth information, referred as
N-view-plus-N-depth
,of
the displayed scenes. There exists algorithms, such as [37] and [38], for extraction
and transmission of 3D scene information via multiple views; hence, as the Inter-
national Organization for Standardization - Moving Picture Experts Group (ISO-
MPEG) standardization activities are to be completed, depth information as well as
the projection matrices of the cameras will all be available for the next generation
LCD panels. Consequently, one should seek for efficient and accurate frame inter-
polation methods specific to multi-view data. The utilization of conventional frame-
rate conversion algorithms for multi-view video increases the amount of data to be
processed by a factor of 2N. Moreover, conventional techniques mainly exploit 2D
motion models, which are only approximations of the 3D rigid body motion model.
In this section, a frame rate conversion system, which estimates the real 3D mo-
tion parameters of rigid bodies in video sequences and performs frame interpolation
for the desired view(s) of the multi-view set, is presented. The main motivation is
firm belief in the utilization of a
true
3D motion model for development of bet-
ter FRUC systems, possibly with much higher frame rate increase ratios. Hence,
in upcoming sections, a completely novel 3D frame-rate up conversion system is
proposed that exploits multi-view video as well as the corresponding dense depth
values for all views and every pixel. The system performs moving object segmenta-
tion and 3D motion estimation in order to perform MCFI. The overall algorithm is
summarized in Fig. 8.
4.1
Depth-Based Moving Object Segmentation
The initial step for the proposed algorithm is the segmentation of the moving rigid
objects, which accounts for independently moving object(s) in the scene. For this
purpose, a depth-based moving object segmentation scheme is utilized. Fig. 9 il-
lustrates typical color and depth frames for
Rotating Cube
and
Akko-Kayo
,[43],
sequences.
The depth maps for
Akko-Kayo
sequence are estimated using the algorithm pro-
posed in [38], whereas those of
Rotating Cube
are generated artificially. The steps
of this segmentation algorithm are provided below:
