Information Technology Reference
In-Depth Information
Fig. 6.
Recall of the correct answer within different
k
positions of the system rank
5.5 Preference Reranker
Our reranking model consists in learning to select the best candidate from a given
candidate set. In order to use SVMs for training a reranker, we applied the Prefer-
ence Kernel Method [13]. In the Preference Kernel approach, the reranking prob-
lem - learning to pick the correct candidate
h
1
from a candidate set
{h
1
,...,h
k
}
- is reduced to a binary classification problem by creating
pairs
: positive train-
ing instances
h
1
,h
2
,...,h
1
,h
k
and negative instances
h
2
,h
1
,...,h
k
,h
1
.
This training set can then be used to train a binary classifier. At classification
time, pairs are not formed (since the correct candidate is not known), while, the
standard one-versus-all binarization method is still applied.
The kernels are then engineered to implicitly represent the
differences
between
the objects in the pairs. If we have a valid kernel
K
over the candidate space
T
, we can construct a preference kernel
P
K
over the space of pairs
T×T
as
follows:
P
K
(
x, y
)=
P
K
(
x
1
,x
2
, y
1
,y
2
)=
K
(
x
1
,y
1
)+
K
(
x
2
,y
2
)
− K
(
x
1
,y
2
)
− K
(
x
2
,y
1
)
,
(5)
where
x, y ∈T×T
. It is easy to show that
P
K
is also a valid Mercer's kernel.
This makes it possible to use kernel methods to train the reranker. The several
kernels defined in the previous section can be used in place of
K
5
in Eq. 5.
6TheExp imen s
We ran several experiments to evaluate the accuracy of our approach for auto-
matic generation and selection of correct SQL queries from NL questions. We
experimented with a well-known dataset GeoQuery developed in order to study
semantic parsing.
5
More precisely, we also multiply
K
for the inverse of rank position.
Search WWH ::
Custom Search