Information Technology Reference
In-Depth Information
Verification and Generalization
The validation and generalization were an important part of the methodology, be-
cause the data was not fully annotated with multiple labels, and there is no inhe-
rent connection between the labels, such as nested hierarchical connection or an
expected logical co-occurrence that could be statistically analyzed.
In order to check the validity of the classifier for a wide range of affective
states, the system was applied to the entire Mind Reading database, which com-
prises 4400 sentences, of 712 affective states, grouped in 24 meaning groups. A
Friedman test was used to check the ranking resulting from the inference results of
all the six sentences that represent each of the affective states. This requires that
all the labels are ranked in a consistent manner. This test yielded significant results
for about 360 affective states.
The Friedman test is not general enough for the case of co-occurring affective
states, because not all the labels can be consistently recognized in all the samples
that represent an affective state, because each sentence represents a variation of
the conveyed concept, as perceived by the speaker, by the annotators, and depends
on the context in which the concept appeared. Therefore, a second threshold was
applied, in order to locate the labels which were consistently recognized and the
labels that consistently were not recognized over the majority of the sentences that
represent an affective state. The threshold for selection was set again at over a
standard deviation above the mean number of sentences, i.e. the double threshold
procedure picked the labels that were selected by at least six machines in at least
four of the six sentences. A complementing criterion was also applied, choosing
the labels that were recognized by less than three machines in at least for of the six
sentences.
Although this seems like statistics over small numbers, the double threshold
was applied to the inference results of the entire Mind Reading database, and
around 570 affective states were consistently characterized with at least one label
of affective-state group. The label combinations inferred by the double threshold
procedure were compared to the lexical definitions of the characterized affective
states, and was found to be correct in over 80 percent of the 570 cases. These are
not small numbers. This analysis also revealed that concepts of affective states
that can be described as combinations of other affective states are also expressed
as combinations of their expressions, in addition to other connections between af-
fective states and their expressions. The distinguishing capabilities of the system
were compared to human performance on an independent test.[60]. The system
that was trained on an English database was later applied to the analysis of sus-
tained human-computer interactions in Hebrew [59]. In this case, the sum of the
pair-wise comparisons was considered directly to monitor the level of the nine af-
fective-state groups along the consecutive utterances. For verification that indeed
the system can be used in this case, test repetitions by each speaker during the in-
teractions were analyzed and were found statistically correlated to events during
the interaction. In addition, simultaneous occurrences were found between the
verbal content of spontaneous speech utterances and the inference results, and be-
tween the inference results and major changes in the values of physiological cues
such as skin conductivity.
Search WWH ::
Custom Search