Interaction Model of Pen-and-Paper User Interfaces - Human-Computer Interaction - page 80

Tabl e 3. 1 Comparison of Core Interactions
Traditional Paper PPUI GUI (following [8])

Writing Inking Text entry

Pointing Clicking Pointing/Clicking

Single sheet

Pointing Reorienting the pen -

Moving paper Moving Dragging

- a Altering shape Altering shape

- a Arranging/Combining Combining

Multiple sheets

- a Subsequent pointing Bridging

a

No core interaction (performed by combining several core interactions)

GUIs incorporate metaphors of traditional desks, they offer somewhat equivalent
interactions for inking, clicking and moving. We deliberately model combining and
bridging as separate core interactions instead of combinations of inking, clicking
and moving. The reason is that one important affordance of paper is to support two-
handed interaction. Both hands can be used to simultaneously interact with two or
more sheets of paper. This stands in contrast to traditional GUIs, which do not sup-
port simultaneous interaction at two or more focus points. Consequentially, combin-
ing and bridging in GUIs should be rather considered as sequential manipulations
of individual areas, hence as a sequence of inking, moving and clicking. Finally,
the interactions of reorienting the pen and altering the shape of paper leverage the
specifics of a pen-and-paper environments and go beyond what is possible in clas-
sical GUIs (pointing, clicking and dragging with the mouse as well as resizing of
windows).
To conclude the discussion of core interactions, we apply them to a representative
set of PPUI systems that support users in interacting with documents. We demon-
strate that these can be classified in terms of the generic core interactions identified
above. Table 3.2 provides an overview of these systems.
A first class of systems (e.g. [32, 165, 135, 78]) augments paper documents by
electronically capturing handwritten annotations ( inking ). ButterflyNet [176] addi-
tionally supports creating associations ( bridging ) between an area of a paper note-
book and a digital photo with a pen gesture. Users can access the digital resource
by tapping with the pen on this paper area. PapierCraft [75] supports tagging paper
documents with pen gestures ( inking ). In a calculator application, Wellner's Digi-
talDesk [167] supports entering numbers by pointing ( clicking ) on a number in an
arbitrary document on the desk, regardless if the document is printed or projected.
Moreover, the position of digital and physical documents on the desk can be mir-
rored to collaborators over distance in real-time. Each time a physical or digital
document is moved , the position is updated at the remote site. FACT [78] and Pen-
Light [138] allow users to select printed content by inking gestures. In addition,
PenLight leverages reorienting the pen in 3d space above paper. By hovering with
the pen and moving it in 3d space, the user can select options from pie menus and
moreover define 3d positions from which a perspective rendering of paper contents
is calculated and projected.