Graphics Reference
In-Depth Information
pixel
(
x
,
y
)
with label
(δ
x
,
δ
y
)
would be
∞
if
(
x
+
δ
x
,
y
+
δ
y
)
was inside the target region
. To perform reshuffling, the data term at pixel
(
x
,
y
)
with label
(δ
x
,
δ
y
)
would be 0
if the user wanted to force
I
(
and infinite for any other shift.
The smoothness term is a typical penalty for label disagreements between neighbors
based on color and gradient mismatches, similar to the discussion in Section
3.3
.
As before,
x
,
y
)
=
I
(
x
+
δ
x
,
y
+
δ
y
)
α
-expansion is used to solve the labeling problem, using a coarse-to-fine
algorithm to make the problem computationally tractable.
3.5.4
Combinations of Methods
Since the explosionof interest in content-aware image retargeting techniques, several
groups have investigated the problem of combining cropping, scaling, non-uniform
warping, and seamcarving to resize images. Dong et al. [
123
] considered seamcarving
followed by scaling, investigating the best intermediate image size at which to make
the transition. Rubinstein et al. [
409
] considered sequences of cropping, scaling, and
seam carving, viewing a candidate sequence as a path through a multidimensional
resizing space. The goal is to find the best path from the original image (represented
as the origin of the space) to a resized image with the desired dimensions (repre-
sented as a hyperplane in the space). Liu et al. [
297
] considered cropping followed
by the non-uniform warping of Wolf et al. mentioned in the next section. An inter-
esting aspect of this work was that the retargeted images were synthesized based on
aesthetic guidelines generally followed by professional photographers — for exam-
ple, the “rule of thirds” specifying that the main subject of a photograph should be
roughly located a third of the distance from each of the horizontal/vertical edges to
the other.
Rubinstein et al. [
407
] presented a comprehensive user study that compared the
retargeting algorithms in this section for the task of reducing image size. The study
suggested that users generally preferred the multi-operator retargeting approach
from the previous paragraph [
409
], the video-motivated algorithm discussed in the
next section [
255
], and—perhaps surprisingly—simplemanual cropping. This paper
also investigated image similarity metrics that correlated with user preferences from
the study.
3.6
VIDEO RECOMPOSITING, INPAINTING,
AND RETARGETING
Wang et al. [
527
] were the first to extend the Poisson gradient-domain image edit-
ing approach to video; the 3D Poisson equation and its discrete approximation are
straightforward generalizations of what we discussed in Section
3.2.1
. This allows
dynamic elements to be composited into a video sequence, such as flickering flames
or a lake with waves and ripples. Since the size of the linear system to be solved is
much larger, fast numerical methods to solve the Poisson equation are critical, as
discussed in Section
3.2.2
.
Many algorithms for video inpainting approach the problem based on the layered
motion model mentioned in Section
2.9
. That is, the video is separated into a static
