Introduction - 3DTV: Processing and Transmission of 3D Video Signals - page 27

View 1 View 2

Synthesized view

Figure 1.10 Multi-view plus depth format video allows synthesis of virtual views
using colour and depth information from nearby viewpoints (Reproduced with
permission of Microsoft)

viewed from a single camera position [44]. Each pixel in the array consists
of colour, luminance, depth, and other data that assists 3D scene rendering.
According to [44], three key characteristics of LDV are:

• It contains multiple layers at each pixel position.

• The distribution of pixels in the back layer is sparse.

• Each pixel has multiple attribute values.

As shown in Figure 1.11, intersecting points between the rays emanating
from a reference camera, and an object are stored [44]. The information
stored for each intersecting point consists of colour and depth information.
The conceptual diagram of a layered depth image is depicted in Figure 1.11
[43, 44]. In Figure 1.11, the intersecting points closest to the reference camera
form the first layer of the LDV. The second nearest are used in the second
layer, and so on. Consequently, each Layered Depth Pixel (LDP) has a
different number of Depth Pixels (DPs), each of which contain colour, depth,
and auxiliary information for reconstruction of arbitrary viewpoints of the
3D scene [44]. For the specific example shown in Figure 1.11, LDP 4 has four
layers, which contain data pertaining to the intersecting points between Ray
D and the object. A single LDI is created per time instant (i.e. the LDI at time
t is composed of the images and depth map values belonging to the same
time instant from all different views).
Conventional video codecs, such as MPEG-4 AVC, are optimized to remove
temporal and spatial redundancies, which are common for most 2D video
sequences. However, the layers of LDV do not exhibit the same characteris-
tics, in terms of spatial and temporal correlation. Conventional video coding
tools are inefficient when coding the layers of an LDV. Therefore, although
LDV is an excellent format for rendering, the lack of an efficient compression
scheme is likely to hold it back.