Information Technology Reference
In-Depth Information
Fig. 3.4
Cross entropy loss
as a function of
f(x
u
)
−
f(x
v
)
To tackle these problems, a new loss function named the
fidelity loss
is proposed
in [
33
], which has the following form:
(
1
−
P
u,v
)
1
P
u,v
(f )
.
P
u,v
P
u,v
(f )
L(f
;
x
u
,x
v
,y
u,v
)
=
1
−
−
−
(3.11)
The fidelity was originally used in quantum physics to measure the difference
between two probabilistic states of a quantum. When being used to measure the dif-
ference between the target probability and the modeled probability, the fidelity loss
has the shape as shown in Fig.
3.5
as a function of
f(x
u
)
−
f(x
v
)
. By comparing the
fidelity loss with the cross entropy loss, we can see that the fidelity loss is bounded
between 0 and 1, and always has a zero minimum. These properties are nicer than
those of the cross entropy loss. On the other hand, however, while the cross entropy
loss is convex, the fidelity loss becomes non-convex. Such a non-convex objective
is more difficult to optimize and one needs to be careful when performing the opti-
mization. Furthermore, the fidelity loss is no longer an upper bound of the pairwise
0-1 loss.
In [
33
], a generalized additive model is proposed as the ranking function, and a
technique similar to Boosting is used to learn the coefficients in the additive model.
In particular, in each iteration, a new weak ranker (e.g., a new feature) is added,
and the combination coefficient is set by considering the gradient of the fidelity
loss with respect to it. When the addition of a new ranker does not bring in signif-
icant reduction of the loss any more, the learning process converges. According to
the experimental results reported in [
33
], FRank outperforms RankNet on several
datasets.
3.2.5 RankBoost
The method of RankBoost [
18
] adopts AdaBoost [
19
] for the classification over
document pairs. The only difference between RankBoost and AdaBoost is that the
distribution in RankBoost is defined on document pairs while that in AdaBoost is
defined on individual documents.
