Information Technology Reference
In-Depth Information
Chapter 8
Support Vector Machine
Vapnik and his research group have studied machine learning based on finite
samples since 1960s. A complete theory, Statistical Learning Theory, has been
established until 1990s (Vapnik, 1995). Moreover a new universal learning
algorithm Support Vector Machine(SVM) has been proposed. SVM minimizes
the probability of classification error based on the structural risk minimization
inductive principle. The main idea of SVM is mapping the nonlinear data to a
higher-dimensional linear space where the data can be linearly classified by
hyperplane (Vapnik et al., 1997). One advantage of SVM is the capacity of
disposing linearity non-separable cases.
8.1 Statistical Learning Problem
8.1.1
Empirical risk
We consider the learning problem as a problem of finding a desired dependence
between input and output (or supervisor's response) using a limited number of
observations. Learning problems could generally be represented as to find
uncertain dependency relationship between variables y and x where the joint
probability distribution function
) is unknown. The selection of the desired
function is based on a training set of n independent and identically distributed
observations:
F
(
x, y
(
x
1
,y
1
), (
x
2
,y
2
),…, (
x
n
,y
n
).
(8.1)
Given a set of functions {
)}
it aims at choosing a best function to
approximate the supervisor's response
f
(
x,w
f
(
x,w
0
)
which makes the risk function
=
Ð
R w
(
)
L y f
( ,
( ,
x w
))
d
F x y
( ,
)
(8.2)
309
