Information Technology Reference
In-Depth Information
intelligent non-player characters that are capable of producing culturally appropriate
behavior in 3D multi-player training scenarios. This discussion will focus primarily
upon the VRP implementation, which is being developed for military mission
rehearsal training. Other Alelo learning environments, such as Operational Dari and
goEnglish, currently employ simplified versions of this architecture, but are migrating
to the new VRP architecture.
5.1 Game Engine Platform
Game engines commonly incorporate features for creating virtual worlds, populating
those worlds with objects and animated characters, and interface controls that enable
users to interact with the objects and the animated characters. We build on these to
create a set of software platforms for building and running virtual environments for
learning culture. These platforms have a common set of capabilities, whether they
support 3D interfaces as in Fig. 1, or 2 ½ D Web-based interfaces as in Figs. 2 and 4.
This is labeled the Game Engine Platform in Fig. 5.
The game engine platform provides a conversational interface that allows the
learners to choose which non-player characters to talk to, and a set of graphical user
interface controls for managing the dialog, as shown in Figs. 2 and 4. These include
buttons for starting and stopping speech input (top right corner in the figures), and
controls for choosing nonverbal gestures for the learner's character to perform. The
interface provides other information such as a summary of the objectives to complete
in the science, hints about what to say and what to do, and a transcript of the dialog.
The game engine platform is also responsible for generating the behavior of each
character. This includes speech, animated gestures, and body movement. We record a
set of possible spoken lines for the characters to say and gesture animations for the
characters to perform, and load them into the game engine platform. The characters
are then able to perform any combination of speech and gesture, as directed by the
social simulation module, which selects the appropriate speech and gestures.
The game engine platform includes a simulation management capability that keeps
track of the location of objects, characters, and landmarks in the virtual world, many
of which have cultural significance. It controls and tracks the movement of people
and objects in the virtual world, and can trigger event notifications in response, e.g.,
cause a non-player to react when a player approaches within a set distance.
5.2 Sociocultural Simulation Module
The Sociocultural Simulation Module is responsible for controlling the behavior of
the non-player characters in the simulation, and ensuring that they behave in
culturally appropriate ways. The agent dialog models follow an interpretation-
decision-behavior cycle, in a variant of the SAIBA agent architecture [16].
The first phase of processing starts with interpreting the learners' spoken
utterances and gestures. We develop speech recognition models for each language,
specially designed for learners of foreign languages. Acoustic models are trained on
both native speech and the speech of learners of the language, to ensure that the
speech recognizer is tolerant of the accents of language learners. The recognizers
utilize grammar-based language models constructed from a database of sample
