Locality Preserving Kernel Hybrid Discriminate Analysis for Dimensional Reduction - Multimedia and Signal Processing - page 16

Locality Preserving Kernel Hybrid Discriminate Analysis
for Dimensional Reduction

Shijin Ren 1,* , Xiaoping Liu 2 , Maoyun Yang 1,2 , and Guiyun Xu 2

1 School of Computer Science & Technology,
Jiangsu Normal University, Xuzhou, Jiangsu 221116, China
{sjren_phd,ymaoyun}@163.com
2 School of Mechanical and Electrical Engineering,
CUMT, Xuzhou, Jiangsu 221116, China
[e-mail], [e-mail]

Abstract. Hybrid discriminant analysis (HDA) combining principal component
analysis (PCA) with linear discriminant analysis (LDA) can achieve better per-
formance for samples following complex distribution. However, HDA can not
work well for complex and nonlinear distributed data. As a result, a locality
preserving HAD (LPKHDA) algorithm is proposed by combining the kernel
method with manifold learning, overcoming the shortcomings of manifold
learning and kernel methods. According to kernel-induced selection criterion,
the optimal kernel parameter of LPKHDA can be achieved efficiently through
gradient method and a boosted LPKHDA algorithm based on Adaboost idea is
implemented. Extensive experiments are conducted to evaluate the proposed
algorithm.

Keywords: Locality preserving, kernel hybrid discriminant analysis, manifold
learning, model selection, kernel-induced space, dimensional reduction.

1 Introduction

Over the past decades, many dimensional reduction methods, such as principal com-
ponent analysis (PCA), linear discriminant analysis (LDA), are applied in many fields
[1-4]. However, PCA might outperform LDA for small sample size (SSS) problem
[3,5], while LDA outperforms PCA in supervised learning. It is necessary to integrate
PCA and LDA into a unified framework. The proposed methods include
PCA+KFDA[6],PCA+LDA[7], hybrid discriminant analysis(HDA) etc [8]. HDA can
extract global and discriminant features from data set, outperforming PCA, LDA and
PCA+LDA[8]. However, HDA model parameters are difficult to be estimated in prac-
tice and may fail to discover more complex and nonlinear relationship that exists in
the data. Both PCA and LDA are not applicable for non-uniformly distributed or im-
balanced data since they have a huge tendency to cluster dominant data set rather than
the small samples.
* Corresponding author.