Biomedical Engineering Reference
In-Depth Information
where vec is a vector stacking operator that creates a vector from a matrix, one column at a time.
Note that the
Z
vector is composed of the previous values of the input that are in the filter taps.
Using this notation, the VAR (
L
) model can be written as
X = AZ + U
(3.3)
Thus, the multivariate least square estimation chooses the estimator that minimizes
(
α ν ν
)
T
Σ
1
−
=
{(
)
T
Σ −
−
1
(
)}
J
=
tr X AZ
−
X AZ
(3.4)
To find the minimum of this function, we have to take the derivative with respect to the
unknown parameters α
∂
∂
J
(
α
α
)
T
−
1
Z
Σ
1
−
=
2
(
ZZ
Σ
⊗
)
α
− ⊗
2
(
)
χ
(3.5)
where
⊗
is the Kronecker product of matrices [
28
] (which creates a matrix whose elements are
given by the elements of the first matrix multiplied by the second matrix). The normal equations are
obtained equating the derivative to zero, and the optimal solution becomes
T
−
1
α
=
((
ZZ
)
Z
⊗
I
)
χ
(3.6)
where
I
is the identity matrix of size
M
. Equation (
3.6
), written back into matrix form, reads
Z
T
T
−
1
A X
=
(
ZZ
)
,
(3.7)
which has the same form as the conventional least square solution of the univariate time series
case. Therefore, we can conclude that the multivariate LS estimation is equivalent to the univariate
least square solution of each one of the
M
equations in (
3.7
), which means that the conventional
adaptive filtering algorithms [
2
] that use least squares can be used by simply renaming the quanti-
ties involved. One can establish the asymptotic normality of the vector LS estimator provided the
data matrix
ZZ
T
is nonsingular, as well as the Gaussianity of the errors in the limit of infinite long
data windows provided the noise is the standard vector white noise (i.e., mean zero, nonsingular
covariance, independent, and with finite kurtosis). It is therefore possible to use the conventional
t
-statistics to check confidence intervals for model coefficients (the number of degrees of freedom
can be estimated by T-ML-1). Furthermore, it can be shown that the vector LS estimator coincides
with the maximum likelihood estimator for Gaussian distributed stochastic processes [
29
].
Unfortunately, the small sample behavior of the multivariate least square estimator is a lot
harder to establish, and normally, Monte Carlo methods are used to establish empiric coefficient
