Image Processing Reference
4 Optical flow
Theoretically, optical flow is the motion of visual features such as points, objects, and shapes
through a continuous view of the environment. It represents the motion of the environment
relative to an observer. James Jerome Gibson first introduced the optical flow concept in the
intended to train the perception of pilots during the war. Perception was considered for the
effect of the motion on the observer. In this context, shape of objects, movement of entities, etc.
are handled for perception. During his study on aviation, he discovered optical flow patterns.
He found that the environment observed by the pilot tends to move away from the landing
point, while the landing point does not move according to the pilot. Therefore, he joined this
concept with the pilot perception on the observed environment.
In the perception of an observer, there may be two options for approaching/departing op-
tical flow around a point. In the first option, the observer may be moving through the target
point. This makes the optical flow departing from the point. In the second option, the envir-
onment around the point may be moving through the motionless observer. This also gives the
same effect, having the optical flow departing from the point. These two options are also valid
for approaching optical flow. If the observer departs from the target point or the point departs
from the motionless observer, the optical flow is seen as approaching through the point.
In video domain, optical flow is commonly known as the apparent motion of brightness pat-
terns in the images. More specifically, it can be conceptualized as the motion of visual features
such as corners, edges, ridges, and textures. through the consecutive frames of a video scene.
Optical flow, here, is materialized by optical flow vectors. An optical flow vector is defined for
a point (pixel) of a video frame. In optical flow estimation of a video frame, selection of “de-
scriptive” points is important. This selection is done using visual features. It is clear that using
an edge point or corner point is more informative than using an ordinary point semantically
as the motion perception of human is based on prominent entities instead of ordinary ones.
Optical flow vectors are, then, the optical flow of video frame feature instances instead of all
Two problems arise in the optical flow estimation of video frames: (1) detection and extrac-
tion of the features to be tracked, (2) calculation of the optical flow vectors of the extracted fea-
tures. Optical flow estimation aims to find effective solutions to these problems. Calculation of
optical flow vectors of the extracted features can be reduced to the following problem; Given a
set of points in a video frame, finding the same points in another frame .
4.1 Derivation of Optical Flow
There are various approaches concerning the estimation of optical flow. Differential, region-
these groups include many algorithms proposed so far. Each of these algorithms reflects the
theoretical background of its group of approach.
Here, the meaning of optical flow estimation is discussed from a differential point of view.
The explanation is based on the change of pixels with respect to time. The solution of the prob-