Bayesian Variable Selection for Multi-response Linear Regression - Technologies and Applications of Artificial Intelligence - page 75

Variable selection methods have also been proposed for group sparsity. For example,
Yuan and Lin (2006) [17] proposed the group Lasso method under the group sparsity as-
sumption. Simon et al. (2013) [13] generalized group Lasso to sparse group lasso. In the
Bayesian framework, Farcomeni (2010) [6] proposed a Bayesian constrained variable
selection approach that can also be used for group selection. Raman et al. (2009) [12]
proposed a Bayesian group Lasso method by extending the standard Bayesian Lasso.
Chen et al. (2014) [3] introduced a Bayesian approach for the sparse group selection
problem.
The linear regression problems treated by the above methods usually involve a single
response vector. In some applications, there can be multiple response vectors, and these
response vectors may be explained by the same or similar subsets of variables to be
selected from a large set of candidate variables. Such shared sparsity pattern enables
different response vectors to collaborate with or to borrow strength from each other to
select the active variables. Such a problem has been studied by Tropp et al. (2006) [16]
under the name of simultaneous sparse coding, where each response vector is a signal,
each predictor vector is a base signal or an atom, and the collection of all the base
signals form a dictionary. The goal is to select a small number of base signals from
the dictionary to represent the observed signals. The problem has also been studied by
Lounici et al. (2009) [9] under the name of multi-task learning, where the regression of
each response vector on the predictor variables is considered a single task. Obozinski et
al. (2011) [10] studied this problem under the name of support union recovery, where
the word “support” means the subset of variables selected for a response vector, and
“support union” means the union of subsets of variables selected for all the response
vectors. If the supports of different response vectors are similar, then the union of the
supports will only be slightly bigger than the supports of individual response vectors.
In this paper, we propose a Bayesian method for solving the above support union
recovery problem, by assuming two nested sets of binary indicator variables. In the first
set of indicators, each indicator is associated with a variable, indicating whether this
variable is active for any of the response vectors. The set of variables whose indicators
are 1's then become the union of the supports. In the second set of indicators, each
indicator is associated with both a variable and a response vector, indicating whether
this variable is active for explaining the particular response vector. So the second set
of indicators gives us the supports of individual response vectors. Variable selection
can then be accomplished by sampling from the posterior distributions of the two sets
of indicators. We develop the Gibbs sampling algorithm for posterior sampling and
demonstrate the performances of the proposed method for both simulated and real data
sets.

2 Problem Setup and the Model

Consider the following multiple response model:

Y = X B + W, (1)

where Y = Y 1 , ··· ,Y M ] is an n × M response matrix of observations, X =

n×p is the fixed n

[ X 1 , ··· ,X p ] ∈ R × p design matrix, B =[ ʲ 1 , ··· ,ʲ M ] is a p × M