Graphics Reference
In-Depth Information
For programs that manipulate images, the choice of image format is almost
always irrelevant: You almost certainly want to represent an image as an array of
double-precision floating-point numbers (or one such array per channel, or per-
haps a single three-index array where the third index selects the channel). The
reason to favor floating-point representations is that we often perform operations
in which adjacent pixel values are averaged; averaging integer or fixed-point val-
ues, especially when it's done repeatedly, may result in unacceptable accumulated
roundoff errors.
There are two exceptions to the “use floating point” rule.
•
If the data associated to each pixel is of a type for which averaging makes
no sense (e.g., an object identifier telling which object is visible at that
pixel—a so-called
object ID channel
), then it is better to store the value in
a form for which arithmetic operations are undefined (such as enumerated
types), as a preventive measure against silly programming errors.
•
If the pixel data will be used in a search procedure, then a fixed-point
representation may make more sense. If, for example, one is going to
look through the image for all pixels whose neighborhoods “look like” the
neighborhood of a given pixel, integer equality tests may make more sense
than floating-point equality tests, which must almost always be imple-
mented as “near-equality” tests (i.e., “Is the difference less than some small
value
?”).
Figure 17.1: An actor, photogra-
phed in front of a green screen,
is to be composited into a scene.
(Jackson
Movie directors often want to film a scene in which actors are in some type of
interesting situation (e.g., a remote country, a spaceship, an exploding building,
etc.). In many cases, it's not practical to have the actors actually be in these situ-
ations (e.g., for insurance reasons it's impossible to arrange for top-paid actors to
stand inside exploding buildings). Hollywood uses a technique called
blue screen-
ing
(see Figure 17.1) to address this. With this technique, the actors are recorded in
a featureless room in which the back wall is of some known color (originally blue,
now often green; we'll use green in our description). From the resultant digital
images, any pixel that's all green is determined to be part of the background; pix-
els that have no green are “actor” pixels and those that are a mix of green and some
other color are “part actor, part background” pixels. Then the interesting situation
(e.g., the exploding building) is also recorded. Finally, the image of the actors is
composited
atop the images of the interesting situation: Every green pixel in the
actor image is replaced by the color of the situation-image pixel; every nongreen
pixel remains. And the partially green pixels are replaced by a combination of a
color extracted from the actor image and the situation image (see Figure 17.2).
The resultant composite appears to show the actor in front of the exploding
building.
Lee/Splash
News/
Corbis)
There are some limitations to this approach: The lighting on the actors does
not come from the lighting in the situation (or it must be carefully choreographed
to approximate it), and things like shadows present real difficulties. Furthermore,
at the part actor, part background pixels, we have to estimate the color that's to be
associated to the actors, and the fraction of coverage. The result is a
foreground
image
and a
mask,
whose pixel values indicate what fraction of the pixel is cov-
ered by foreground content: A pixel containing an actor has mask value 1; a pixel
Figure 17.2: The actor, compos-
ited atop an outdoor scene. The
detail shows how the horse's tail
obscures part of the background,
while
some
background
shows
through.
(Jackson
Lee/Splash
News/Corbis)
