Geoscience Reference
In-Depth Information
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.2
Unlabeled data
Positive data
Negative data
True boundary
SVM boundary
S3VM boundary
0.1
0.1
0
0
−0.1
−0.1
−0.2
−0.2
−0.3
−0.3
−0.4
−0.4
−0.5
−0.5
−0.4 −0.2
0
0.2
0.4
0.6
−0.4 −0.2
0
0.2
0.4
0.6
x 1
x 1
(a) S3VM in local minimum
(b) S3VM in “wrong” low density region
Figure 6.5: (a) With labeled data in adjacent quadrants, S3VM comes close to the true boundary, but it
is sensitive to local minima (small gaps between unlabeled data from sampling noise). (b) With labeled
data located in opposite quadrants, the S3VM decision boundary seeks a gap in the unlabeled data. In
this case, the gap is orthogonal to the classes, so S3VM suffers a large loss.
To obtain a quantitative understanding of the significance of these problems, we sampled
5000 random datasets of this form with l
10000. In roughly half the cases, both
labeled instances are in the same class, and both a standard SVM and S3VM label everything the
same class and get 50% error. In one quarter of the cases, the first problem above occurs, while in
the other quarter of the cases, we face the second problem. On average among the half of cases with
both classes represented, SVM achieves an unlabeled (transductive) error rate of 0 . 26 ± 0 . 13, while
S3VM performs much worse with an error rate of 0 . 34
=
2 and u
=
0 . 19. (These are mean numbers, plus or
minus one standard deviation, over the 5000 random datasets.) The 8% difference in error shows
that S3VMs are not well suited to problems of this kind where the classes are not well-separated,
and a low-density region does not correspond with the true decision boundary. In fact, despite the
large variance in the error rates, S3VM is statistically significantly worse than SVM in this scenario
(paired t -test p
±
0 . 05).
This chapter has introduced the state-of-the-art SVM classifier and its semi-supervised coun-
terpart. Unlike the previous semi-supervised learning techniques we discussed, S3VMs look for a
low-density gap in unlabeled data to place the decision boundary. We also introduced entropy regu-
larization, which shares this intuition in a probabilistic framework based on logistic regression. This
is the final chapter introducing a new semi-supervised learning approach. In the next chapter, we
explore the connections between semi-supervised learning in humans and machines, and discuss the
potential impact semi-supervised learning research can have on the cognitive science field.
Search WWH ::




Custom Search