Image Compositing and Editing - Computer Vision for Visual Effects - page 74

“warping in” background texture from images taken from different perspectives (see
Section 3.7 ). In this section, we focus on what can be done when a clean plate or an
approximation to one is unavailable, and we only have the pixels in the current frame
as the basis for inpainting.
Most inpainting algorithms iteratively fill in the target region from its boundary

∂

inward; in this context is sometimes called the fill front . We initialize the process

with

I

( x , y )( x , y ) ∈

I 0 ( x , y ) = (3.28)

black ( x , y ) ∈

Then at each step we produce a new image I n + 1 ( x , y ) based on I n ( x , y ) , until the

hole is filled in.
Here, we discuss two basic approaches to the problem: a partial-differential-
equation-based approach better suited to inpainting thin holes, and a texture-
synthesis approach better suited to inpainting large missing regions.

3.4.1 Partial-Differential-Equation-Based Methods

Bertalmio et al. [ 40 ] were originally inspired by the way that professional art conser-
vators restore damaged paintings. The basic approach is to work inward from the
boundary , propagating the image colors and structure from the surrounding pix-

els into the hole. In this section, we assume that each color channel is processed
independently.
Weneed toanswer twoquestions tomake this approachmoreprecise. First, exactly
what information should be smoothly propagated into the hole? Second, in what
direction should the propagation occur? Bertalmio et al. proposed to propagate the
Laplacian of the image intensities, since this operator represents the local smooth-
ness of the image. While it would seem natural to propagate the Laplacian in the
direction normal to the hole boundary

∂

∂

, this can lead to counterintuitive results
in the presence of strong image structure, as illustrated in Figure 3.17 b. Instead, we
propagate along isophote directions — that is, along contours where the intensity
change is the smallest — producing a more natural result (Figure 3.17 c). Essentially,
we try to continue strong image edges into the hole.

Ω

(a) (b) (c)

Figure 3.17. (a) The image and inpainting region. (b) Propagating image information normal to
the region boundary can produce unwanted artifacts. (c) Instead, we propagate along isophote
directions to maintain visual continuity.