Graphics Reference
In-Depth Information
pipelines. Note the initial processing differs but eventually merges at
the behavior analysis.
3.2 Processing the virtual human's utterances
The utterance pipeline (left side in Figure 4) analyzes the surface text of
the virtual human's utterance to infer appropriate nonverbal behavior.
The processing in NVBG does not make any strong assumption about
the input's markup of the agent's communicative intent or internal
state (e.g., affective state, attitude). When such information is missing,
the system attempts to infer it (essentially falling back on the more
limited role of illustrating and embellishing the language channel).
For instance, in the absence of detailed markup of the virtual human's
communicative intent, such as points of emphasis or emotion, NVBG
analyzes the surface text to support the generation of believable
nonverbal behaviors.
To this end, the sentence is first parsed to derive the syntactic
structure. Then a semantic analysis phase attempts to infer aspects of
the utterance's communicative function using inference rules to build
up a hierarchical structured lexical, semantic and pragmatic analysis.
Examples of these communicative functions include affirmation,
inclusivity, intensification, etc. (Lee and Marsella (2006) for details).
NVBG then goes through a behavior analysis stage, in which a
set of
nonverbal behavior rules
map from communicative functions
to classes of nonverbal behaviors. A BML generation phase then
maps those behavior classes to specific behaviors, described in BML.
This mapping can use character-specific mappings designed to
support differences including personality, culture, gender and body
types. Conflict resolution occurs at several phases in the overall
process. For example, if there are two or more rules overlapping with
each other causing conflict, NVBG resolves the conflict by filtering
out the rule with lower priority. The priority value of rules has been
set through a study of human behaviors using video corpora. The
final result is a schedule of behaviors that is passed to the character
animation system.
Research on NVBG has explored several approaches to encoding
the knowledge used in the function derivation and behavior mapping.
Initial work on NVBG was based on an extensive literature review of
the research on nonverbal behavior. This seeded the development of
rules encoding the function derivation and behavior mapping rules.
Then videos of real human face-to-face interactions were annotated
and analyzed to verify the rule knowledge, embellish knowledge with
dynamic information about behaviors and develop a conflict resolution
Search WWH ::
Custom Search