Graphics Reference
In-Depth Information
Fig. 7.13
Future AR application using 3D sensing.
Image sources
www.hao-li.com
,
www.pcl.org
and
www.3ders.org
7.3 Challenges in AR
As described in the earlier sections, AR applications hold great promise for mobile
users in the near future but mobile devices cannot yet deliver on this promise. But to
implement the technologies discussed earlier, hardware (HW) architectures have to
evolve in parallel to provide efficient resources that can keep power consumption at
an acceptable level. One answer is an embedded heterogeneous system (HMP) with
highly specialized HW blocks and dedicated data buses and memory architectures,
like the AR Engine, hardware IP designed by Metaio [
6
,
17
,
18
]. Although an HMP
is a great solution on the HW level, it has to be complemented by intelligent program-
ming frameworks for scheduling and resource management. Combining optimized
tracking technologies with efficient HW IP and easy-to-use software development
tools is the foremost challenge of the decade for AR, and has to be solved to ensure
seamless application development for various AR applications and across multiple
mobile platforms.
7.3.1 AR Hardware IP (AR Engine)
As shown in Fig.
7.14
, feature extraction, descriptor building, matching, and
tracking are the key hotspots in the AR pipeline. The most critical prerequisite for
any AR application is the exact knowledge of the camera pose (pose being the combi-
nation of position and viewing direction, together 6 degrees of freedom (DOF)). This
pose defines a fixed coordinate system in the real world that can be used for seamless
rendering of content from the virtual world. If the camera pose is not known from
other sources, it has to be determined by computer vision techniques. Figure
7.14
shows a typical AR pipeline. In the first half of the pipeline, the captured images are
processed for camera pose estimation. In the second half, the camera pose is used
