Information Technology Reference
In-Depth Information
2.1.3 Capturing Touch Input
A further important field for bridging paper and computers is to track touch input
on physical sheets of paper. The most frequent approach for detecting touch input
on physical paper consists of observing the scene with an overhead camera. Touch
events are detected in the camera image using image processing techniques. An in-
fluential early example is Wellner's DigitalDesk [167]. The DigitalDesk recognizes
the position of fingers by their shape in the video frames. A microphone attached to
the bottom of the desk is used to detect the exact moment when the user's finger taps
on the desk. However this does not allow the system to detect multiple touches that
occur simultaneously. Another technique can detect multiple simultaneous touches.
It tracks the color of the fingertips. When the color changes, it is assumed that the
finger is pressed against a surface [93]. Further approaches analyze shadows cast
by the fingers [171] or use depth information for detecting whether a finger is hov-
ering over or is touching the surface. Wilson [172] recently presented a technique
that uses a depth camera to detect touches not only on flat, but also on curved sur-
faces. Figure 2.4 shows how the approach works: The scene (left) is observed by a
depth camera. Initially a model of the surface is set up that stores the depth of the
surface for each pixel while no hands or fingers are visible in the camera image.
The current input of the depth camera (center picture) is then compared against this
model. Hands and fingers have lower depth values because they are situated above
the surface. If the distance between the value stored in the model and the currently
recorded value falls below a certain threshold, a contact is detected (right).
A drawback of camera-based capturing is that it significantly restricts the mo-
bility of paper. Typically, the camera is mounted at a fix position. While there exist
mobile solutions, such as the Docklamp [55] or Sixth Sense [96], these are still
rather large and heavy and their use is not comparable to the flexibility of tradi-
tional pen and paper. This restriction can be alleviated by using multiple cameras
that capture a large volume [173].
Other approaches for detecting touch use electromagnetic field sensing [6] or
embed electronics into paper, e.g. push buttons [114, 9].
Fig. 2.4
Detecting touch using a depth camera. Left: Touching a topic. Center: Depth image. Right:
Detected contacts (photos courtesy of Andy Wilson)
