Other Applications: Sequential Dependence Modelling and Data Mining - On Statistical Pattern Recognition in Independent Component Analysis Mixture Modelling - page 153

7.1 Including Sequential Dependence in ICAMM

7.1.1 Introduction

Mixtures of independent components analyzers are progressively recognized as
powerful tools for versatile modelling of arbitrary data densities [ 1 - 7 ]. In most
cases, the final goal is to classify the observed data vector x (feature) in a given
class from a finite set of possible classes. To this aim, the probability of every class
given the observed data vector p ½ C k = x must to be determined. Then the class
having maximum probability is selected. Bayes theorem is claimed for the prac-
tical computation of the required probabilities since it allows expressing p ½ C k = x in
terms of the vector observation mass density. Considering K classes, we can write

p ½ C k = x ¼ p ½ x = C k p ½ C k
p ½ x

p ½ x = C k p ½ C k
P k 0 ¼ 1 p ½ x = C k 0 p ½ C k 0 ; ¼ ð 7 : 1 Þ

where the mixture model of p ½ x is evident in the denominator of Eq. ( 7.1 ). The
ICA mixture model (ICAMM) considers that the observations corresponding to a
given class k are obtained by linear transformation of vectors having independent
components plus a bias term: x ¼ A k s k þ b k : Equivalently, this implies that the
observation vector in a given class can be expanded around a centroid vector b k in
a basis formed by the columns of A k . It is assumed that the basis components are
independent so that the A k matrix is nonsingular. When this assumption becomes
invalid, due, for example, to a high dimension of the observation vector, some
dimension reduction techniques like classical PCA are routinely used. The trans-
forming matrix, the centroid, and the marginal probability density functions
(which can be arbitrary) of the independent components of s k (called sources)
define a particular class.
Using standard results from probability theory, we have that p ½ x = C k ¼
j det A k j p ½ s k : On the other hand, algorithms for learning the ICAMM parameters
ð A k ; b k ; p ½ s k k ¼ 1...K Þ in supervised or unsupervised frameworks can be found
in the given references [ 1 - 7 ]. Therefore, if the classifier has been trained, we can
compute the required probabilities using

j det A k j p ½ s k p ½ C k
P k 0 ¼ 1 j det A 1

k 0 j p ½ s k 0 p ½ C k 0 s k ¼ A 1

p ½ C k = x ¼ ð x b k Þ: ð 7 : 2 Þ

k

However, the classes and observations very often do not appear in a totally
random manner, and they exhibit some degree of sequential dependence in time or
space domains. This means that the computation of the class probabilities should
consider the whole history of observations. Thus, if we define the indexed matrix
of observations X ð n Þ x ð 0 Þ x ð 1 Þ ...x ð n ½ ; we should compute pC k ð n Þ= X ð n ½ :
In Sect. 7.1.2 , we present a procedure to extend ICAMM to the case of
sequential dependence (SICAMM). In Sect. 7.1.3 , some experiments are included