Information Technology Reference
In-Depth Information
large camera arrays [16]. Video technologies that can handle such high frame rates
have been opening up a new era in video applications. The Vision Chip architecture
[17] realizes a high-speed real-time vision system on a integrated VLSI chip, and
has been applied to robotic systems. A real time tracking system that uses a high
speed camera has been studied[18]. In addition, an optical flow algorithm for high
frame rate sequences has been investigated [19].
Since high frame-rate video requires stronger encoding than low frame-rate
video, the statistical properties of high frame-rate video must be elaborated so as
to raise encoding efficiency. In particular, it is important to have an accurate grasp
of the relationship between frame-rate and bit-rate. When the frame-rate increases,
the correlation between successive frames increases. It is easily predicted that in-
creasing the frame-rate decreases the encoded bits of inter-frame prediction error.
However, the quantitative effect of frame-rate on bit-rate has not been fully clari-
fied. The modeling of inter-frame prediction error was tackled by [20]. The deriva-
tion processes are sophisticated and the results are highly suggestive. Regrettably,
however, the model does not consider the effect of frame-rate on prediction er-
ror. Modeling the relationship between frame-rate and bit-rate was addressed by
the pioneering work of [21] . They assume that some asymptotic characteristics
hold, and then inductively generate an interesting model. Unfortunately, however,
the model does not consider the effect of motion compensation on the predic-
tion error. In other words, the model covers the bit-rate of the inter-frame predic-
tion error without motion compensation. It is important to consider the inter-frame
prediction error with motion compensation, since representative video coding al-
gorithms like H.264/AVC and MPEG-2 adopt inter-frame prediction with motion
compensation.
This chapter establishes mathematical models of the relationship between frame-
rate and bit-rate in anticipation of encoding high frame-rate video. These models
quantify the impact of frame-rate on the bit-rate of inter-frame prediction error
with motion compensation. The exact nature of the relationship depends on how
the frame-rate is converted. We consider two frame-rate conversion methods. The
first one is temporal sub-sampling of the original sequences, that is frame skip, as
shown in Figure 2. The shaded rectangles represent frames at each frame-rate and
rectangles enclosed by dotted-line represent the down-sampled frames. Figure 2 (b)
and (c) illustrate the sequences sub-sampled to 1/2 frame-rate and 1/4 frame-rate,
respectively, by subsampling the original sequence shown in Figure 2 (a). The sec-
ond one is a down-sample filter based on average operator. When the open interval
of the shutter in the image pickup apparatus increases, motion blur occurs, which
is known as the integral phenomenon. The integral phenomenon changes the sta-
tistical properties of the video signal. This integral phenomenon can be formulated
as a mean filter. Henceforth, the first model is called temporal sub-sampling and its
output sequences are called temporally sub-sampled sequences. The second model
is called temporal down-sampling and its output sequences are called temporally
down-sampled sequences.
