Digital Signal Processing Reference
In-Depth Information
Chapter 3
Semantic Object Segmentation
Xiaogang Wang
Abstract
Semantic object segmentation is to label each pixel in an image or a video
sequence to one of the object classes with semantic meanings. It has drawn a lot of
research interest because of its wide applications to image and video search, editing
and compression. It is a very challenging problem because a large number of object
classes need to be distinguished and there is a large visual variability within each
object class. In order to successfully segment objects, local appearance of objects,
local consistency between labels of neighboring pixels, and long-range contextual
information in an image need to be integrated under a unified framework. Such inte-
gration can be achieved using conditional random fields. Conditional random fields
are discriminative models. Although they can learn the models of object classes
more accurately and efficiently, they require training examples labeled at pixel-level
and the labeling cost is expensive. The models of object classes can be learned
with different levels of supervision. In some applications, such as web-based im-
age and video search, a large number of object classes need to be modeled and
therefore unsupervised learning or semi-supervised learning is preferred. Therefore
some generative models, such as topic models, are used in object segmentation be-
cause of their capability to learn the object classes without supervision or with weak
supervision of less labeling work. We will overview different technologies used in
each step of the semantic object segmentation pipeline and discuss major challenges
for each step. We will focus on conditional random fields and topic models, which
are two types of frameworks widely used in semantic object segmentation. In video
segmentation, we summarize and compare the frameworks of Markov random fields
and conditional random fields, which are the representative models of the generative
and discriminative approaches respectively.
X. Wang (
)
Department of Electronic Engineering, The Chinese University of Hong Kong,
Hong Kong, China
e-mail:
xgwang@ee.cuhk.edu.hk
