Dense 3D Reconstruction and Tracking of Dynamic Surface - Advances in Image and Graphics Technologies - page 214

weight. Marker-less methods is a very attractive non-invasive approach since it is not
restricted to motion information associated with markers. However, for both marker-
based and marker-less techniques, most of them provide surface reconstruction
without spatio-temporal coherence which is usually important in many applications.
Recently, some researchers focus on optical flow method to reconstruct and track
the deformable surface. For example, [9] provided an optical-flow based approach for
deformable surface tracking using a mesh based deformation model together with
smoothing constraints that force the mesh to shrink instead of fold in presence of self-
occlusion, and [10] used optical flow to estimate scene flow from a calibrated stereo
image sequence. Despite of their success in obtaining dense data of dynamic surface,
they are easy to be influenced by illumination, and the process is usually involved in
intensive computation.
Some other approaches [15, 16] tracked the deformable surface represented by a
triangulated mesh where the problem is formulated as Second Order Cone
Programming (SOCP). Though these methods can obtain good results, some prior
knowledge of the possible deformations, such as the pose in the first or each frame of
the deformable surface, should be required.
In this essay, we present a novel spatio-temporal framework of dense 3D
reconstruction and tracking of dynamic surface from a calibrated stereo image
sequence using block matching, by extending the Lucas-Kanade method [2] into the
spatio-temporal domain. In our approach, the surface in one image can be divided into
several blocks, and the counterparts of each block can be found by optimizing an
energy function in the stereo and temporal images. Through this way the block
correspondences of whole image can be obtained, so the surface can be reconstructed
and tracked densely. However, due to the different parameters in different
neighboring blocks, gaps may exist when we put together different reconstructed
blocks. To solve this problem and get a smooth and dense surface, a bilinear
interpolation method is adopted.

2 The Proposed Method

2.1 Description of the Notations

There are a lot of notations (listed in the following table) used in this research paper.

2.2 Divide One Surface into Several Blocks

We consider four images in a stereo image sequence, two images in the left image
sequence and two in the right image sequence. The previous left image is called
template image because it is the benchmark image for finding the correspondences in
the four images, and the other three images will project back to the template image.
One surface in template image can be divided into several blocks for every block