Information Technology Reference
In-Depth Information
However, accurate measurement of depth information from a natural scene still
remains problematic due to the difficulty of depth estimation on textureless, depth
discontinuous and occluded regions.
In general, we can classify depth estimation methods into two categories: active
depth sensing and passive depth sensing. Passive depth sensing methods calculate
depth information indirectly from 2D images [4]. Contrarily, active depth sensing
methods employ physical sensors for depth acquisition, such as laser sensors, infrared
(IR) sensors [5], or light pattern sensors [6]. Although current direct depth estimation
tools are expensive and produce only low-resolution depth images, they can obtain
more accurate depth information in a shorter time than passive depth sensing methods.
We can obtain depth images from a natural scene in real time using an IR-based
time-of-flight (TOF) depth camera, such as Z-Cam, developed by 3DV Systems,
Ltd. [7] or NHK Axi-vision HDTV camera [8]. The depth camera simultaneously
captures color images and the associated depth images by integrating a high-speed
pulsed IR light source with a conventional video camera. The ATTEST project
has shown us a possibility of realizing a 3D TV system using a depth camera [9].
In addition, 3D contents generated by a depth camera were demonstrated for fu-
ture broadcasting [10].
In spite of these successful activities using a depth camera, we still suffer from
handling depth information captured by current depth cameras due to their inherent
problems. The first problem is that
a depth image captured by the depth camera
usually includes severe noise
. This noise usually occurs as a result of differences in
reflectivity of IR sensors according to color variation in objects. The second prob-
lem is that
the measuring distance of the depth camera to get depth information
from a real scene is limited
. In practice, the measuring distance is approximately
from 1m to 4m. Thus, we cannot obtain depth information from far objects. The
last problem is that
the current depth camera can only produce low-resolution
depth images
. Most depth cameras usually have a resolution of less than 320×240
depth pixels due to many challenges in real-time distance measuring systems. The
maximum image resolution of depth images acquired by Z-Cam is 720×486.
In order to solve these built-in problems, we introduce a system to generate
high-quality and high-resolution video-plus-depth by combining a high-resolution
stereoscopic camera and a low-resolution depth camera, called a
depth camera-
based hybrid camera system
or a
hybrid camera system
shortly [11]. There are
three questions to be addressed in this chapter related to the hybrid camera system:
1) How can we obtain high-quality and high-resolution video-plus-depth us-
ing a hybrid camera system?
2) How can we render consecutive 3D scenes generated by high-resolution
video-plus-depth rapidly using a mesh representation?
3) How can we stream 3D video contents including video-plus-depth data and
computer graphic images?
In this chapter, we first introduce a method to obtain high-quality and high-resolution
video-plus-depth using a hybrid camera system [12]. The hybrid camera system pro-
vides region-of-interest (ROI) enhanced depth images by regarding depth information
captured by a depth camera as ROI depth information on the left image captured by a
