Information Technology Reference
In-Depth Information
with the auxiliary task are called the “source-domain data”. Evidently, the source-
domain data cannot be utilized directly in training a ranking model for the target
domain due to different data distributions. And this is exactly what the technology
of transfer learning wants to address.
Some preliminary attempts on transfer ranking have been made in recent years,
such as [
1
] and [
4
]. Here, we take [
1
] as an example to illustrate the detailed process
of transfer ranking. In particular, the authors of [
1
] have studied the problem of
transfer ranking from two aspects, i.e., feature level and instance level, respectively.
9.1 Feature-Level Transfer Ranking
The
feature-level transfer ranking
method assumes that there exists a low-dimen-
sional feature representation (may be transformations of the original features) shared
by both source-domain and target-domain data. Specifically, suppose that these
common features are the linear combinations of the original features,
z
k
=
u
k
x,
(9.1)
where
x
is the original feature vector and
u
k
is the regression parameters for the new
feature
z
k
.Weuse
U
to denote the matrix whose columns correspond to the vectors
u
k
.
Suppose the scoring function
f
is linear, that is,
f(z)
=
α
T
z
, where
z
is the
learned common features. As a result, we have
f(z)
=
α
T
Ux
=
w
T
x
, where
w
=
αU
T
.Let
A
=[
α
s
,α
t
]
, where
s
and
t
stand for source domain
and target domain, respectively, then we have
W
=
UA
.
The next step is to minimize the losses for both the source and target domains, in
a manner of multi-objective optimization, in order to learn
U
and
A
simultaneously.
Intuitively, according to the assumption that the source domain and target domain
share a few common features,
A
should have some rows which are identically equal
to zero. In order to reflect this intuition, a
(
2
,
1
)
-regularization item for
A
is added,
which is obtained by first computing the
L
2
norm of the rows of matrix
A
to form
a new vector, and then computing the
L
1
norm of this vector. It has been shown in
[
1
] that although this optimization problem itself is non-convex, it can be converted
to an equivalent convex optimization problem and get effectively solved.
According to the experimental results in [
1
], the feature-level transfer ranking
method can gain improvements over baselines, such as Ranking SVM [
2
,
3
].
, and
W
=[
w
s
,w
t
]
9.2 Instance-Level Transfer Ranking
The
instance-level transfer ranking
method makes use of the source-domain data by
adapting each of them to the target domain from a probabilistic distribution point of
view. Technically, this is implemented via instance re-weighting. Suppose the source
