Graphics Reference
In-Depth Information
Images at different
viewing and
lighting directions
Filter Bank
Figure 9.6
Filtering of BTF data to construct textons. Each filter is applied to each image (left), which
multiplies the data (right). (After [Leung and Malik 01].)
Because the variation in lighting and viewing direction affects the local ap-
pearance of a surface, the filter response is more complicated in the 3D case.
The texton creation starts with a set of images captured under different light-
ing/viewing directions, i.e., a BTF data set. The images are arranged so that each
pixel corresponds to a single sample point on the surface sample. Each filter in
the filter bank is applied at each pixel of each image. A filter applied to an en-
tire image is just another image called a “filter image.” Figure 9.6 illustrates this
schematically. There is one filter image for each filter in the bank, and one set of
filter images for each image in the BTF data set. In Figure 9.6, the filter-response
vectors for a single image can be viewed as 1D vertical sections of the filter image
stacks at the right.
The next step is to cluster the filter-response vectors into 3D textons. This is
illustrated schematically in Figure 9.7. For each pixel, the filter-response vectors
of all the images are concatenated into a single vector: if there are 48 filters
and
n
images, then the filter-response vectors have 48
n
elements. The response
vector for sample point therefore contains the responses to all the filters in all the
lighting/viewing combinations available in the BTF data set. Clustering is done
using the
k
-means algorithm, which produces a set of clusters corresponding to
the 3D textons. Again, the idea of clustering is that all the response vectors in
each cluster are essentially equivalent, and the center response vector serves as
the representative.
Each vector is assigned to the cluster such that the distance from the center of
the cluster (the norm of the difference of vectors) is minimized. For sample points
assigned to the same cluster, their appearances in the small local neighborhood
have similar characteristics under the different viewing and lighting directions.
Therefore, every vector assigned to the same cluster is approximated by the rep-
