Applications of High-Performance Computing to Functional Magnetic Resonance Imaging (fMRI) Data - High-Throughput Image Reconstruction and Analysis - page 267

is
significantly smaller than the total number of coefficients ( N k )), then one can use
techniques of sparse regression to find a reliable solution to (12.3).

If the model is sparse (i.e., the number of nonzero coefficients in

{

a ij ( t ) }

12.2.2 Sparse Regression
Consider a multivariate linear regression model of the form

Y = ZW

× ×

where Y is a known n 1 1 response vector, Z is a known n 1 n 2 regressor matrix,

× and W is the unknown model vector of size n 2 1tobedeterminedusingthe

response Y and regressor Z . Such a problem is typically solved with techniques
such as ordinary least square regression, ridge regression, or subset selection [21].
For these techniques to work, a general requirement is that n 1 >>

n 2 . However,
recent research shows that if W is sparse then it may be recovered even if n 2 > n 1

using the lasso regression [6, 18].
The lasso regression [18] solves the problem

2
2

|| − ||

mi W Y ZW (12.6)

|| || ≤

s.t. W t (12.7)

1

2

|| || where

2 represents the sum of squares of the coefficients of X (square of L2
norm of X )and

X

|| ||

1 represents the sum of absolute values of the coefficients of
X (its L1 norm). The parameter t is the regression parameter usually chosen after
cross-validation. In the extreme case, when t is infinitesimally small, the optimal
solution W corresponds to the regressor (column of Z ) which is ''closest to'' Y .In
the other extreme when t is sufficiently large, (12.7) ceases to be a constraint and
the optimal solution W corresponds to classical the least square solution of (12.3).
It can be verified that for any t ,thereexistsal such that the program (12.6),
(12.7) is equivalent to the following optimization problem:

X

2
2

|| − || | |

mi W Y ZW + l W (12.8)

1

The programs (12.6), (12.7), and (12.8) can be solved efficiently using a tech-
nique called least angle regression [6].

12.2.3 Solving Multivariate Autoregressive Model Using Lasso
The estimation of multivariate autoregressive coefficients in (12.3) may be viewed
as a regression problem where Y is the response variable, Z is the matrix containing
the regressors, and W is the model to be determined. In this case, the maximum
likelihood estimate of (12.4) becomes the least square solution to the regression
problem. The lasso formulation thus becomes