Expert-Based Fusion Algorithm of an Ensemble of Anomaly Detection Algorithms - Technologies and Applications of Artificial Intelligence - page 114

Expert-Based Fusion Algorithm of an Ensemble
of Anomaly Detection Algorithms
Esther David 1 ,GuyLeshem 1 , Michal Chalamish 1 , Alvin Chiang 2 ,
and Dana Shapira 3
[e-mail], [e-mail],
{ michal.chalamish,alvin.chiang.180,shapird } @gmail.com
1 Department of Computer Science, Ashkelon Academic College, Ashkelon, Israel
2 Department of Computer Science and Information Engineering, National Taiwan
University of Science and Technology, Taiwan
3 Department of Computer Science, Ariel University, Israel
Abstract. Data fusion systems are widely used in various areas such
as sensor networks, robotics, video and image processing, and intelligent
system design. Data fusion is a technology that enables the process of
combining information from several sources in order to form a unified
picture or a decision. Today, anomaly detection algorithms (ADAs) are
in use in a wide variety of applications (e.g. cyber security systems, etc.).
In particular, in this research we focus on the process of integrating the
output of multiple ADAs that perform within a particular domain. More
specifically, we propose a two stage fusion process, which is based on
the expertise of the individual ADA that is derived in the first step. The
main idea of the proposed method is to identify multiple types of outliers
and to find a set of expert outlier detection algorithms for each type. We
propose to use semi-supervised methods. Preliminary experiments for
the single-type outlier case are provided where we show that our method
outperforms other benchmark methods that exist in the literature.
Keywords: Anomaly Detection Algorithms, Cluster, Ensemble, Out-
lier, Scores.
1 Introduction
According to the current state of the art, a wide range of anomaly detection
algorithms (ADA) are proposed in various disciplines such as statistics, data
mining, machine learning, information theory and spectral decomposition [1],
which are also known as outlier detection algorithms [2]. Given the decisions of
multiple ADAs, which all operate in the same environment, in this research we
aim to confront the challenge of integrating the individual decisions into a final
unified representative decision. Specifically, we are interested in non-stationary
(i.e., unstable and unexpected) environments where the algorithms improve the
decision making process using partial feedback given to them sporadically (that
is, at unknown times) and the correctness of the feedback is also unknown [18].