Information Technology Reference
In-Depth Information
Thus, in the proposed algorithm, the root level of the data is analyzed by
means of standard NNSC. NNSC can effectively find the optimal non-negative
bases and then reproduce the data as the combination of sparse hidden compo-
nents involving linear additive interactions.
Assume that non-negative input data X is a matrix, then linear decomposition
describes the data as X
AS. The matrix A contains as its columns the basis
vectors of the decomposition. The rows of matrix S contain the corresponding
hidden components that give the contribution of each basis vector in the input
vectors.
Establishing the objective function as follow:
≈
2
+
λ
ij
F
(
A,S
)=
1
2
X
−
AS
S
ij
(1)
To get the small reconstruction error with the sparseness, the function must be
minimized with the constraints
∀
ij
:
A
ij
0
,S
ij
0and
∀
i
:
a
i
=1
,
where
a
i
denotes the i:th column of A.
λ
(
0) is the sparseness-regulatory parameter
to control the tradeoff between sparseness and accurate reconstruction.
Then, through constantly updating the basis matrix A and hidden components
matrix S until the objective function reaching the minimum, the optimal A and
S can be achieved.
Hence, in the root level, each data can be represented as linear additive com-
bination of a few bases and then grouped into a certain cluster corresponding to
the basis which gives the largest contribution to the data.
≥
2.2 Deep Learning of the Basis for the Subsequent Level
In the root level, all the data have been clustered, but the training experiments
show that there is a large size-gap among the maximum and minimum clusters.
Commonly, the data with several different dimensions are still grouped into a
same cluster. But the basis corresponding to the cluster cant describe all the
related data especially for the ones with low projection values on the basis. As
a result, these data lack adequate descriptions so that the local representation
of the whole dataset is limited and not detailed. Therefore, its desired to have
deeper analysis of relatively big clusters by further analyzing and expanding the
corresponding bases.
Moreover, the sparseness of the response of the data based on expanded bases
should not be invariant. The more intense response of the data in certain base,
the more potentially the data contains the sub-level information of the base,
when these data are more accurately described. Thus, to more precisely represent
the data locally near the certain base, the sparseness of these data are accordingly
higher.
a) First step is to expand the number of bases to make each basis composed
of several sub-bases. Because these sub-bases can have more detailed description
of the data especially for the ones with poor representation in the upper level.
Search WWH ::
Custom Search