Graphics Reference
In-Depth Information
things; substituting a pen for the mouse's pointing functionality is often reason-
able (although if clicking or double-clicking is part of the process, then the sub-
stitution may have to be more complex, with pen taps replacing button clicks).
Replacing the pen with a Wiimote, or with your hand in a Kinect system, is sim-
ilarly reasonable, although with each substitution, the details of the interaction
must necessarily change. What
doesn't
change is the intent to identify or select
certain objects in the scene through some interaction, which makes the separation
of intent from implementation a natural boundary.
In interaction, there is communication between human and computer, typi-
cally in two languages: The user-to-computer direction involves various interac-
tion devices, and the computer-to-user direction is primarily through the display
to the eye, although there may also be audio or touch components. The mean-
ing and form of each of these languages constitute natural abstraction boundaries:
We must decide what things a user may communicate to the computer (meaning)
and how each thing is communicated (form), and vice versa. There is also a third
component: the relationship of interaction device to display, or the mathematics or
algorithm required to transform the input into something meaningful in the output.
But this is typically application-dependent and represents the
computation
rather
than the communication between human and machine.
The two languages in turn break down into finer levels.
•
Conceptual design
is the model of the user's understanding of the appli-
cation (e.g., a 3D modeling application), typically consisting of objects
(shape, texture, control point), relationships among objects (textures are
applied to shapes, splines are governed by control points), and operations
on them (we can apply a texture to an object, or reshape a spline curve).
•
Functional design
is the specification of the interface to the operations
of the conceptual design. It includes a specification of what information
is needed for an operation, what errors may occur (and how they are to
be handled), and what the results are. The functional design is an abstrac-
tion of the operations, but
not
of the user interface. We would specify that
to apply a texture to a shape, we need the texture and the shape and the
texture coordinates on the shape, but would leave the question of how the
user communicates the texture or shape to a later stage. Conceptual and
functional design together constitute the “meaning” part of the interaction
language.
•
Sequencing design
describes the ordering of inputs and outputs, and the
rules by which inputs may be assembled to generate meaning. A click and
drag on a model may be meaningful (indicating screen-aligned translation
of the model), while a click and drag on the empty part of a menu bar may
be ignored as meaningless.
•
Lexical design
determines what constitutes the units of a sequence. For
input, these are things like a single click, a double-click, a drag operation,
etc. For output, they may be things like blinking, displaying a dialog box,
the choice of font or text color for text display, etc.
Not all interaction is purely sequential; in two-handed multitouch interfaces,
both hands may be doing things that, taken together over some period of time, have
some meaning, but the precise ordering is irrelevant; nonetheless, a generalized
notion of sequence design provides a good boundary even in these cases.
