A Look into the Future: Augmented reality (AR) and Android TV - Pro Android Games - page 314

Using a Single Surface for Camera and GL Drawing
It turns out that is indeed possible to use a single surface to draw both camera video
and OpenGL. As mentioned, the setPreviewDisplay (SurfaceHolder holder) method
of the Camera API sets the surface to be used for live preview. Nonetheless, there is
also the method setPreviewTexture (SurfaceTexture surfaceTexture) , which sets a
SurfaceTexture to be used for preview. SurfaceTexture captures frames from an image
stream as an OpenGL ES texture. The image stream may come from either camera preview
or video decode. A SurfaceTexture may be used in place of a SurfaceHolder when
specifying the output destination of a Camera object. Doing so will cause all the frames from
the image stream to be sent to the SurfaceTexture object rather than to the device's display.
There are two limitations when using a SurfaceTexture :
The texture object underneath

SurfaceTexture uses the GL_TEXTURE_
EXTERNAL_OES texture target, which is defined by the GL_OES_EGL_
image_external OpenGL ES extension. This limits how the texture
may be used. Each time the texture is bound, it must be bound to the
GL_TEXTURE_EXTERNAL_OES target rather than the GL_TEXTURE_2D target.
Additionally, any OpenGL ES 2.0 shader that samples from the texture
must declare its use of this extension using, for example, an
#extension GL_OES_EGL_image_external : require directive. Such
shaders must also access the texture using the samplerExternalOES
GLSL sampler type.
 setPreviewTexture and SurfaceTexture are only available in API level 11
(Android 3.2) or later. This eliminates most of the Android devices out
there (perhaps more than 90%). Thus, at this point in time, it is simply
not feasible to use SurfaceTexture . Nevertheless, when Android 4
becomes the dominant version, I think SurfaceTexture will be the
standard for capturing and processing video.



Drawing and Application Life Cycle
Drawing and the application life cycle are the second and third hurdles of using the ARTK
in Android. As mentioned, the PC version of the ARTK uses OpenGL to draw. AndAR
overcomes this by using a second surface for OpenGL ES drawing, as explained in the
previous section. The tricky part is coordinating drawing between the two concurrent
threads and using the ATK API calls to detect the markers and obtain a transformation
matrix required to bind the virtual object to the marker.