Eyes and Ears - Learn to Play: Designing Tutorials for Video Games - page 94

This is by no means new information. Education scholars, psycholo-
gists, and cognitive scientists have known for a while now that the
mind processes audio and video on different channels. My first intro-
duction to this concept came from Paivio's dual coding theory, which
has been mentioned throughout. Dual coding theory states that all
data are stored as either a word or a picture. Ergo, we store our mental
“data,” if you will, in the form of images and sounds, or at least verbal
representations of sounds. Under this theory, if I'm able to get you to
remember both the image and the sound of the instructions on how to
play a game, you will be more likely to remember it later than if I only
gave you one of those two things. This isn't always strictly sound and
pictures; for example, the word gun and a picture of a gun are fine as
an image and a “sound,” even if they are both on paper. To clarify this
mix-up, Paivio called the two things imagens and logogens to differenti-
ate them as pictures and words, and not strictly sounds and sights.
Other theories deal with working memory. Working memory is
the part of your brain that is actively working on sensory input to
make sense of it, relate it to existing schema or make new ones as
necessary, and then add it to your long-term memory stores. We know
that this is limited by a few things that we've discussed, like cognitive
load and the magic number.
As the various theories wrought in years of academic work have
shown us, and as I discussed in Chapter 3, there is much debate about
how the mind processes and stores information. Despite these aca-
demic disagreements, however, there are a few things about memory
and information processing that we know really well.
First, sound and audio have their own dedicated processing cen-
ters (see Figure 6.1). These are referred to in Baddeley and later as
the visuospatial sketchpad for visual traces and the phonological loop
for auditory traces. These act like a sound studio and an art studio.
Phonological things like words and sounds are recorded and processed
as sound and associated to and from schema appropriately. Visual
things, much the same, are processed and associated where they are
needed in the schemata of the long-term memory. Because these two
parts of the mind are distinct, all information brought in from video
and sound are processed separately; if the two are presented simul-
taneously, they must be separated first and processed second, which
incurs load.