Cognitive Dialog Systems for Dynamic Environments: Progress and Challenges - Digital Signal Processing for In-Vehicle Systems and Safety - page 129

Information State Update paradigm [ 9 ]. The interaction state also comprises
variables that describe the detected user state to allow adaptive selection of speech
acts based on the user's current situation.
The system is also able to switch its behavior between different styles for the
realization of one selected speech act, depending on the user's state. Different
behavior styles can change the processing of speech acts in many aspects.
For example, the content of a speech act realization can differ in its length and
complexity based on the user's workload. It is also possible to adjust the speaking
speed, the volume of the voice, and the stress of certain key phrases according to
this parameter. Using those parameters, the co-driver realizes a verbose, chatty, and
entertaining behavior if it detects a state of low cognitive workload. It presents
much information, tells occasional jokes, and shows expressive mimic.
For situations with high cognitive workload, the co-driver switches to a different,
more concise, and unobtrusive behavior to use the limited available cognitive
resources for the transmission of the most critical information. In this style, the
system also takes more initiative in the interaction, taking most noncritical

decisions from the user.
A user study [ 10 ] showed that a behavior which adapts to the changing user's
cognitive load is both more efficient and also more satisfying for the user than a
nonadaptive one. By changing the information throughput depending on the
workload level, the system can optimally use the available cognitive resources of
the user without risking overload. This behavior was evaluated as empathic and
desirable by the users in a satisfaction questionnaire. It is therefore critical for a
cognitive interaction system to provide this kind of adaptation.

8.5 Recording Setup

During the interaction, we employ a variety of signals to observe the user in the car.
This is done for multiple reasons. First, an adaptive dialog system needs online data
streams from which it can extract meaningful features describing the user's state.
Second, to train automatic recognizers that perform this user state classification, we
need to provide large amounts of labeled training data. To that end, we installed
multiple biosignal sensors in the car to get a reliable, continuous data stream
without obstructing or distracting the user too much.
We employ the following equipment to observe the user:

￿ Small cameras to record videos of the face and the upper body of the driver to
catch facial expressions and body pose.
￿ A close-talking microphone to record the user's utterances
￿ Brain activity is measured using electroencephalography (EEG) with one of two
possible alternatives:

- A 16-electrode EEG cap with active electrodes for optimal signal quality and
coverage of all brain regions
- A 14-electrode gaming device (Epoc Emotiv) with saline electrodes for
increased usability and reduced setup time