Communication Aspects: How Landmarks Enrich the Communication Between Human and Machine - Landmarks: GIScience for Intelligent Services - page 193

information from the kiosk system [ 19 ] . This is an indicator for efficient dialog
handling. More importantly, the overall user satisfaction was 90 %. The reason may
be that almost 90 % of the participants found the test location eventually, and about
80 % of those with no or only small wayfinding problems (i.e., only minor confusion
along the way, but without really taking wrong turns) [ 19 ] .
Dethlefs et al. [ 20 ] evaluated their approach in a computer-based survey without
actual wayfinding. Participants had to rate turn-by-turn route directions and destina-
tion descriptions generated by their approach, descriptions produced by a human
direction giver, and by Google Maps (no destination descriptions from Google
Maps; no commercial system is capable of producing them). Participants were
asked to determine which of the presented instructions were automatically generated
by a computer and which instructions appeared to be most useful. Instructions
generated by Google Maps were clearly identified to be computer-based (94 % of
participants). For the descriptions generated by Dethlefs et al.'s approach, only
36 % were classified as computer-generated, thus, 64 % of the participants took
them as being from a human communication partner. This applies to turn-by-
turn directions. Forty-two percent of the destination descriptions were correctly
identified as computer-generated, but also 34 % of the human directions were falsely
taken to be from a machine. In conclusion, this approach produces instructions that
appear to be more natural than those generated by Google Maps, and may, thus, be
better suited for human-computer interaction.
However, in terms of usefulness ratings results tell a different story. While
destination descriptions by Dethlefs et al.'s approach are rated by both familiar
and unfamiliar participants to be more useful than the human-generated ones (53
vs. 46 %; 65 vs. 33 %, respectively) turn-by-turn directions are only perceived as
most useful by 7 % of the familiar participants. Human directions were seen to be
most useful here, just ahead of those by Google Maps (48 vs. 42 %). Unfamiliar
users are more positive; 37 % prefer Dethlefs et al.'s directions. Familiar users
may reject them because the instructions tend to be verbose and, as discussed in
Sect. 6.2.1 , may include odd landmark references. The significant caveat with this
study is that participants did not actually have to find their way. The study only tested
for naturalness (where their approach is strong) and user preference (where it is
popular with unfamiliar participants, but not so much with familiar ones). No actual
performance data has been collected, therefore, no statements regarding which kind
of directions is actually better can be made.
The SpaceBook system [ 38 ] was evaluated in a setup where participants had to
perform eight tasks in two runs. These included both navigation and tourist infor-
mation tasks. The system was tested against a baseline system relying on standard
smartphone applications. Participants rated both systems equally successful in terms
of task completion. However, the baseline system had a better task completion rate
in most navigation tasks, whereas the SpaceBook system performed better in the
tourist information tasks. This is also reflected in user preferences. The base system
is preferred for navigation tasks, the SpaceBook system for tourist information tasks.
With the SpaceBook system, users had major issues with navigation because the
system did not provide any graphical information, i.e., no map or directional arrows.