Graphics Reference
In-Depth Information
Fig. 7.12
Examples where 3D depth sensing would improve AR applications.
Image sources
structure.io
as shown in Fig.
7.12
. With 3D depth sensing readily available one can perform (a)
precise measurements of the environment, (b) creation of a 3D model of the phys-
ical spaces, (c) extract accurate object scale and shape information, (d) occlusion
handling so that AR content can be seamlessly integrated into the physical world,
(e) generation of CAD models for 3D printing, and (f) easy content generation for
the cloud in the form of cad models to perform 3D object recognition, matching,
tracking, rendering, etc. As discussed above, depth cameras will enable three major
features for seamless AR experience:
1.
Easy content generation
2.
Easy mapping and measurement of the real world
3.
More robust detection and tracking of the real worlds
.
In the coming years, multiple 3D sensors will be introduced in the mobile space
similar to Kinect-like cameras for smartphones and tablets and the benefits of this
technology will then be available for an average user. The Google “project Tango” is
one of the examples [
10
]. The arrival of depth-based cameras in a mobile device will
enable a quantum leap in AR user experience. The next interesting sensor technology
that comes right after depth cameras are array cameras. Arrays of CMOS image
sensors, either in a 2
4ormoreasthe
technology matures will enable computational photography in really tiny form factor
featuring all focus, low light, and depth video streams which can further enhance AR
use experience. Unfortunately, point cloud processing has a large footprint in terms
of memory, power, and computation. Power consumption, in particular, of 3D depth
sensing and of those arrays, together with the required additional image processing
will be a major challenge that needs to be addressed and hence hardware acceleration
of key hot spots in the depth sensing pipeline would help in alleviating this challenge
(Fig.
7.13
).
×
2 form factor at the beginning, and 4
×
