Empowering People Rather Than Connecting Them - Human-Computer Interaction and Innovation in Handheld, Mobile and Wearable Technologies - page 273

devices. This section is an attempt to highlight
some of the interesting research challenges we
need to solve in order to make progress.

film industry, where complex physical mod-
els are now controlled by artists with little
need for them to understand the underlying
physical models. In mobile software engi-
neering, we therefore need to be enabling
interaction designers to create dynamics, the
way animators create films. This is likely to
be done via standard parameterised primi-
tives, using a range of physical metaphors,
with accompanying vibration and audio
feedback profiles. The quality of output is
open-ended - the models can progressively
improve as computational power and models
improve, and as better sensing and displays
evolve. This will then lead to the evolution
of richer cultures of linking such models to
abstract data structures.
5. Real-time mobile Operating Systems:
For a sense of flowing, embodied control,
we will need mobile devices with real-time
architectures, with hard real-time guarantees.
This will involve tight loops from sensing
to actuation on device (including fast vibro-
tactile and audio responses, which are not
subject to random delays due to multi-thread
operating systems, as in current popular
platforms). It will also need guaranteed low-
latency round-trip-times over the wireless
network for interaction between people at a
distance. This would lead to a major rethink
of mobile operating systems requirements.
6. Power implications of control & inference
loops: This issue combines aspects of the
'midas touch' challenge, and the real-time
mobile challenge, with hardware and soft-
ware design and the choice of interaction
metaphors. Without appropriate hardware
and algorithms for intelligent allocation
of limited processing power, none of these
techniques will be used in everyday mobile
devices.
7. Aesthetic-utilitarian trade-off: Will inter-
actions be so rewarding people enjoy work-
ing hard to 'master' them? E.g. In Mobile

1. The Midas touch: How do we control the
interpretation of our phone's sensor read-
ings? How do we 'declutch' certain modes?
The information flow from the sensors will
need be interpreted differently in different
contexts. This requires excellent models to
automatically infer likely intention given
overt behaviour, and we need subtle feedback
to the user for them to infer current mode
and consequences of action. This is a major,
fundamental area which will recur every-
where in mobile multimodal interaction.
2. Inference, learning and adaptation:
Understanding the data generated by mobile
users: To survive in mobile service provi-
sion, companies will need the skills to create
models, infer latent variables, such as context
and goals, and be able to negotiate these
interpretations with the users. Interpretation
of context, of music content, of emotional
aspects of behaviour and the meaning of
interactions within social networks.
3. Uncertainty: Every interaction involves un-
certainty. An interface should be honest—it
must work with the uncertainty and not just
filter it out blindly, as is the norm today.
Increasing computational power makes
this ad hoc approach less defendable. The
quality of control of a system depends on
its feedback. The feedback must reflect the
uncertainty of system beliefs. Appropriate
use of uncertain feedback can lead to
smoother interaction, with user behaviour
regularised appropriately (e.g. Körding &
Wolpert, 2004).
4. Physical simulation for interaction design:
Physical simulation based models for real-
time synthesis for audio and haptic rendering.
The model for this could be the development
of the tools within the computer-generated