Geoscience Reference
In-Depth Information
6.1
SUPPORTVECTORMACHINES
We first discuss SVMs. Our discussion is intended to be just enough for the purpose of introducing
S3VMs in the next section. For a complete exposition see any standard textbook, e.g., [
49
,
149
,
176
].
For simplicity, we will assume that there are two classes:
y
∈{−
1
,
1
}
. We will also assume that the
D
. Such a decision boundary is defined by the set
decision boundary is linear in
R
w
x
{
|
+
b
=
}
,
x
0
(6.1)
D
is the parameter vector that specifies the orientation and scale of the decision bound-
ary, and
b
∈ R
where
w
∈ R
=
(
1
,
1
)
1, the decision
boundary is shown as the blue line in Figure 6.2. The decision boundary is always perpendicular to
the vector
w
. The value
b
determines the shift along
w
.
is an offset parameter. For example, when
w
and
b
=−
w=(1, 1)
1
0.5
w'x+b=0
0.707
0
0.5
1
Figure 6.2:
The linear decision boundary (the blue line)
w
x
(
1
,
1
)
+
=
=
b
0, where
w
(the red
1. The distance from point
(
0
,
0
)
to this decision boundary is 1
/
√
2 (the green line).
vector), and
b
=−
w
x
0. We will predict
the label of
x
by sign
(f (
x
))
. We are interested in the distance between an instance
x
to the decision
boundary. The absolute value of this distance turns out to be
Let
f(
x
)
=
+
b
. The decision boundary is thus defined by
f(
x
)
=
|
f(
x
)
|
/
w
. For example, the origin
has a distance 1
/
√
2
≈
0
.
707 to the decision boundary, as shown by the green line in
=
(
0
,
0
)
x
Figure 6.2.
The decision boundary cuts the feature space into two halves, one half with
f>
0 (the positive
side), and the other half with
f<
0 (the negative side). We define the
signed distance
of a labeled
instance
(
x
,y)
to the decision boundary as
yf (
x
)/
w
.
(6.2)
The signed distance is positive, if a positive instance is on the positive side, or a negative instance
on the negative side. For now, we also assume that the training sample is
linearly separable
, meaning
that there is at least one linear decision boundary that can separate all labeled instances so they are
