Graphics Reference
In-Depth Information
brightness is locally fairly constant, and if the set of “edges” in each image are in
the same locations. The notion of “locally” depends on how the image is viewed:
If each pixel subtends 1
◦
at the eye, “locally” may mean “over a region a few
pixels wide,” while if each pixel subtends 0. 01
◦
, “locally” may mean “several
hundred pixels.” Indeed, it's possible to make images that appear similar at one
distance but distinct at another distance. A simple example is a black-and-white
checkerboard pattern and a gray rectangle: Close up, they're quite distinct; at a
large distance (great enough that the visual system cannot distinguish the indi-
vidual checkerboard squares) they appear identical. More complex examples (see
Figure 5.3) are described by Oliva [OTS06].
Applications.
How much does all this matter for graphics? Since much of
graphics is used to make people say that they are seeing some particular thing
when they view their computer's display, it's quite important. On the other hand,
our understanding of the visual system is still relatively sparse, so adapting our
imagery to affect perception at the retinal level may be relatively easy, while try-
ing to adjust it to affect the way in which whole objects are perceived may be
more challenging and more prone to unexpected results. Furthermore, there's an
interaction between
low-level vision
(the parts of our visual system responsible
for detecting things like rapid changes in brightness in a particular area, typically
the early-vision parts) and
high-level vision
(the parts responsible for forming
hypotheses like “I'm seeing a surface with a pattern on it”) that is still not well
understood. Mumford, in an essay on pattern theory [Mum02], cites a remark-
able analogous example from the auditory system: Psychologists recorded various
sentences—“The heel is on the shoe,” “The wheel is on the car,” “The peel is on
the orange”—and replaced the first phoneme of the second word in each sentence
with noise, resulting in “The #eel is on the shoe,” for example, where the hash
mark denotes noise. Subjects who listened to these sentences perceived not the
noise-replaced sentences, but the originals, and indeed, did not notice a phoneme
was missing. Thus, as Mumford notes, the actual auditory signal did not reach
consciousness. On the other hand, the replacement phoneme could only be deter-
mined from the larger context of the sentence. Mumford conjectures that vision
may, in many cases, work the same way: While low-level information is often
extracted from what you see, in some cases the way in which it's treated may be
influenced by the results of higher-level understandings that you get from partially
assembling the low-level information. For example, when you see someone lean-
ing against a railing, you form the hypothesis that the railing continues behind the
person, without ever consciously considering it. When you see something through
the spinning blades of a fan, you assemble the parts you see at different times into
a coherent whole, rather than assuming that the obscured parts at one instant are
unrelated to the unobscured parts in the same portion of your visual field a moment
later. Because of these interactions between high-level and low-level vision, we'll
concentrate primarily on the low-level aspects, which are better understood.
Do we really “see” things? It's more accurate to say that our visual system
contructs a model of the world from its input, forming this model with a combina-
tion of perceptual and cognitive processes that resolve apparent contradictions in
the perceptual data (as in the experiment Mumford describes). This lets the brain
eventually form an object hypthesis (“I see
this
thing!”), albeit with some back-
tracking if your cognitive abilities contradict what you think you saw (“That
can't
be a flying elephant!”). Thus, the end result of vision is a construction created by
the mind, and not objective reality.
Figure 5.3: Close up, you see
Einstein; from a distance, you
see Marilyn Monroe. (Image
courtesy of Aude Oliva, MIT.)
