Geoscience Reference
In-Depth Information
5
5
4
4
3
3
2
2
1
1
0
0
−5 −4 −3 −2 −1
0
1
2
3
4
5
−5 −4 −3 −2 −1
0
1
2
3
4
5
yf (
x
)
f (
x
)
(a) the hinge loss
(b) the hat loss
y(
w
x
Figure 6.3:
(a) The hinge loss
c(
x
,y,f(
x
))
=
max
(
1
−
+
b),
0
)
as a function of
yf (
x
)
. (b) The
w
x
hat loss
c(
x
, y,f(
x
))
=
max
(
1
−|
+
b
|
,
0
)
as a function of
f(
x
)
.
SEMI-SUPERVISED SUPPORTVECTORMACHINES
∗
6.2
Semi-Supervised Support Vector Machines (S3VMs) were originally called Transductive Support
Vector Machines (TSVMs), because its theory was developed to give performance bounds (theo-
retical guarantees) on the given unlabeled sample. However, since the learned function
f
naturally
applies to unseen test instances, it is more appropriate to call them S3VMs.
Recall that in Figure 6.1(b), the intuition of S3VM is to place
both
labeled and unlabeled
instances outside the margin. We have seen how this can be encouraged for the labeled instances
using the hinge loss in Figure 6.3(a). But what about unlabeled instances? Without a label, we do
not even know whether an unlabeled instance
x
is on the correct or the wrong side of the decision
boundary.
Here is one way to incorporate the unlabeled instance
x
into learning. Recall the label predic-
tion on
x
is
sign
(f (
x
))
. If we treat this prediction as the
putative label
of
x
, then we can apply
the hinge loss function on
x
:
y
=
−
y(
w
x
c(
x
, y,f(
x
))
=
max
(
1
+
b),
0
)
sign
(
w
x
+
b)(
w
x
=
max
(
1
−
+
b),
0
)
w
x
=
max
(
1
−|
+
b
|
,
0
),
(6.15)
where we used the fact sign
(z)z
. This new loss function is distinct from the hinge loss in that it
does not need the real label
y
, but is instead completely determined by
f(
x
)
. The new loss function
is plotted in Figure 6.3(b). Note the
x
-axis is now
f(
x
)
instead of
yf (
x
)
. Due to its distinctive shape,
this new loss function in (6.15) is called the
hat loss
.
Because of the way we generate the putative label
=|
z
|
y
, an unlabeled instance
x
is
always
on
the correct side of the decision boundary. Nonetheless, the hat loss still penalizes certain unlabeled
