Touch and Speech: Multimodal Interaction for Elderly Persons - Biomedical Engineering Systems and Technologies - page 372

is the proportion of times that the actual data agree with the attribute values, including
the system failed-understanding situation as not matched, and
P(E) = ∑

is the proportion of times that the actual data are expected to be agreed by chance,
where M(i, M) is the value of the matched cell of row i, M(i) the sum of the cells of
row i, and T the sum of all cells.
Therefore, we summarized the results of all the tasks and constructed one confu-
sion matrix for all the data, and got: kappa coefficient κ = 0.88(std.=0.10) for touch
and κ = 0.74 (std.=0.13) for speech modality. This in general still suggests a success-
ful degree of interaction using touch and speech modality. However, touch input is
performing more effectively at the detailed interaction level compared to the speech
modality, due to the common problem caused by the automatic speech recognizer
(294 SNU, 19.6 averagely for each participants).

6.2 Efficiency of MIGSEP

Regarding the efficiency of MIGSEP, the automatically logged quantitative data are
summarized in table 3, with respect to user turns, system turns and the elapsed time
for each participant and each task.

Table 3. Quantitative results calculated based on the recorded data concerning efficiency

Touch Speech

Mean Std. Mean Std.

User turns 15.5 (7) 4.1 4.3 (3) 1.7

Sys turns 15.4 (5) 3.9 4.3 (3) 1.6

Elapsed time (s) 88.9 40.2 57.6 24.2

From a general interaction perspective, a very good overall performance efficiency
is shown by averagely 4.3 user turns and 4.3 system turns per task for each participant
using speech modality, because the average basic turn numbers, inferred by the
shortest solution for each task to be filled, are 3 user turns and 3 system turns. This
indicates that almost every participant (std. < 2) was able to find the shortest way to
complete the tasks while tolerating the problem of automatic speech recognizer.
However, 15.5 user turns and 15.4 system turns compared with the shortest solution 7
and 5 respectively were less convincing for the touch modality. Given further insight
into the individual data, four participants were having much more turns (averagely
21.8 user turns) than the others, since they were slightly lost finding certain targets
and started to do unreasonable brute-force searching.
On the other hand, the elapsed time for each task and each participant for both
modalities is considered as satisfying: with averagely 88.9 second for theoretically
minimal 12 interaction paces (7+5) with touch input, which is only 7.4 second each,
and 57.5 second for minimal 6 interaction paces (3+3), which is only 9.6 second each.
However, the standard deviation of 40.2 for touch input is a bit high, due to the inte-
raction context unawareness of the four individuals with 144.8 second averagely con-
sumed time.