Information Technology Reference
In-Depth Information
Si
nce
f
(
ʲ
)
is continuously differentiable, tending
ʵ
ₒ
0 gives us
(
s
k
)
j
≤
ʻ/
√
|
G
k
|
=
,...,
|
G
k
|
for
j
1
. This implies that
⊛
⊝
j
⊞
1
/
2
2
⊠
s
k
2
≤
G
k
ʻ
/
|
G
k
|
=
ʻ.
(14.7)
∈
On the other hand, suppose that
ʲ
G
k
=
0
is optimal. From (
14.6
), we get
1
ʻ
ʲ
G
k
2
+
2
2
2
s
k
=
ʲ
G
k
⃒
ʲ
G
k
2
+
2
ʻ
ʲ
G
k
2
+
ʻ
−
s
k
2
=
0
.
Since
ʲ
G
k
2
≥
0 we obtain
ʲ
G
k
2
=
s
k
2
−
ʻ
. Replacing
ʲ
G
k
2
above and
rearranging terms, we have
1
s
k
.
ʻ
ʲ
G
k
=
−
s
k
2
Combining with the condition in (
14.7
), the claim follows.
As we can see, the ratio between
ʻ
and the norm of a partial gradient vector
s
k
2
determines whether to select the corresponding group (
s
k
2
>ʻ
) or not
(
s
k
2
≤
ʻ
acts like a threshold for choosing groups.
It is also worthwhile to note that the optimal subvector
). That is, the value of
ʻ
ʲ
G
k
for a selected group
G
k
is not necessarily sparse, since there is no mechanism that pushes its coefficients
to the zero values, as we can see from the theorem above. Therefore group lasso may
not be appropriate for finding features not only groupwise but also within groups,
for instance to detect alternative splicing events from exon microarrays.
14.2.1.2 Scaling of Regularization
In group lasso (
14.4
), we can consider rescaled versions of the regularizer
ʨ
G
,
K
ʨ
G
(
ʲ
)
=
ʻ
w
k
ʲ
G
k
2
,
k
=
1
where a scaling factor
w
k
is defined for each group
k
=
1
,
2
,...,
K
. Scaling factors
can be defined according to applications. Two examples include:
w
k
=
√
|
•
ʲ
G
k
2
in (
14.4
) penalizes the square root
of the degree of freedom to present a group
G
k
. This scaling can be used to penalize
degrees of freedom, not square roots of them [
16
,
24
]. This setting is suitable when
multinomi
al va
riables are represented as groups of dummy variables.
G
k
|
: without scaling, each term
/
√
|
(
√
|
•
w
k
=
,
and therefore larger groups tend to be penalized more than smaller ones in (
14.4
).
1
G
k
|
: without scaling, the order of magnitude of
ʲ
G
k
2
is
O
G
k
|
)
Search WWH ::
Custom Search