Graphics Reference
In-Depth Information
or temperature. Synesthesia may be induced with
drugs. The most common types of synesthesia is
the word-to-color association, and colored hear-
ing, when someone is seeing colors when hearing
a musical tone, usually the lower a musical note,
the darker the color. Some musicians interpret
their color experience in terms of musical keys
or associate their mood or feeling with musical
keys. Therefore synesthesia can be used as a
metaphor for transposing elements indigenous to
one medium into another (Ox, 1999).
Synesthetic ways of expression are applied in
electronic art, performance arts, data presentation,
technical implementations, design, advertisement,
visualizations and simulations for scientific and
educational purposes; they address and interact
with the varied audience. Several writers, such
as Edgar Allan Poe or Arthur Rimbaud described
synesthetic experience and even linked it with
memory. Some poets state they hear sounds or
musical tones when they see words, images, and
colors, and musicians assert that they see colors
assigned to particular tones or musical passages.
Interactive multimodal data presentations are of
interests to scientists. Many kinds of user interfaces
are realized in visual, auditory, or tactile domains.
The haptic/touch interfaces, for example pressure
sensitive interfaces, exemplify another kind of
multimodal data presentations.
Digital images, sounds, or animations can be
analyzed as metaphors and seen as important sign
systems working beyond the literary culture. Some
artists create synesthetic metaphors that cross the
sense modalities; they may seem incomprehen-
sible, for example when someone (probably a
musician) said that, in a mood of irritability, he
bought a coat in the color of C sharp minor (Grey,
2000). The reduced division between the culture of
the image and that of the text, for example, sending
messages containing words, images, sounds and
objects, makes synesthetic modes of cognition
operate outside the language-based structures of
thinking. Messages conveyed in the non-verbal
mode of visual communication media are often
being more abstract and more interactive with the
viewer. By crossing senses, viewers construct their
perception of the reality that is communicated by
an artist in their own art-historical and cultural
contexts. Strong aesthetic experience elicited
by synesthetic art may result in a sensation of
increased artistic potential and a need for doing
one's own artwork. Examples of the approaches
to visual presentation of music and sound include
artwork sonification and environmental problems
sonification. Examples of scientific sonifications
include a pie chart sonification enabling the
hearer to understand information, sonification of
ocean buoy data (Sturm, 2005), or sonification
of proteins (Dunn & Clark, 1999). Van Scoy and
Gifu (2004) generated music by converting the
relations in data sets in order to alert a viewer to
the existence of hidden clusters of such kinds of
data as mathematical functions and basketball
data. The haptic/touch interfaces, for example
pressure sensitive interfaces exemplify another
kind of multimodal data presentations.
See Table 3 for Your Response.
SPATIAL PERCEPTION AND
COGNITION
Spatial skills and abilities mean competence in
spatial visualization or orientation. Spatial vi-
sualization is an ability to mentally rotate, twist,
or invert pictorially presented objects. Spatial
orientation is the understanding of the array of
objects, utilizing body orientation of the observer.
When we recognize something, we compare what
we see with our memory representation. First, we
build (encode) the structural schema of what we
see in both visual and verbal memory, and make
comparison to the canonical objects (those with
already known top/bottom, front/back, and left/
right differences). The split-brain research results
(when cerebral hemispheres become discon-
nected) show that spatial information is common
to both hemispheres.
