Information Technology Reference
In-Depth Information
be minimal. Where {
f
(
x,w
)} is the set of expected functions,
w
is functional
general parameters.
L
(
y
,
f
(
x,w
)) measures the loss, or discrepancy between the
response
) provided
by the learning machine. Different type of learning problems have diverse formal
loss function.
Empirical risk minimization inductive principle is used in the classical
methods of learning problem. Empirical risk defined on the basis of the training
set.
y
of the supervisor to a given input
x
and the response
f
(
x, w
1
l
= Ã
R
(
w
)
L y
(
,
f
(
x
,
w
))
(8.3)
e
mp
i
i
l
i
=
1
Machine learning designs learning algorithm for minimizing R emp (
w
).
8.1.2 VC Dimension
Statistical learning theory is inductive learning's theory on the basis of small
sample size. One significant concept is VC Dimension (Vapnik-Chervonenkis
Dimension) in pattern recognition. The VC dimension of a set of indicator
functions
) is equal to the largest number h of vectors that can be separated
into different classes in all the 2 h possible ways using this set of functions (i.e.,
the VC dimension is the maximum number of vectors that can be shattered by the
set of functions). The VC dimension is equal to infinity if there exists a set of
vectors that any number of samples can be shattered by the functions
f
(
x,w
). The
VC dimensions of a set of bounded real functions is defined by transformed
indicated functions with threshould.
The VC dimension of the set of functions (rather than the number of
parameters) is responsible for the generalization ability of learning machine.
Intuitively, learning machines with high VC dimensions are more complexity and
power. At the moment there is no universal theory to calculate VC dimension of
an arbitrary function set. In n dimensions real space Rn the VC dimension of
linear classification and linear real function is
f
(
x,w
n
+1. However, The VC dimension
of
) are infinite. In statistical learning theory, it is still a problem
that how to calculate VC dimension using theory and experimental method.
f
(
x
, α)=sin(α x
Search WWH ::




Custom Search