Beyond Aggregation - Real-Time Analytics - page 493

change,andminimizetheoccurrencesoffalsealarms.Thisismoregenerally
known as anomaly detection.
Outlier Detection
The simplest form of anomaly detection is outlier detection. Outliers are
observations that fall far away from their expected values, but in such a way
that the process producing the data remains unchanged. In a fixed dataset,
it is usually fairly easy to identify the outliers with methods ranging from
visual inspection to goodness-of-fit methods. These methods usually do not
translate well to the online setting, where the most common approach is to
use a threshold mechanism to identify outliers.
The online versions of these techniques rely on the deviation of the value
from its predicted value. The primary difference between them is the metric
they use to decide that a value is truly an outlier. Regardless, they rely on
the implementation of a forecaster, which could be simply defined by this
interface:
public interface Forecaster {
public double forecast(double y);
}
Any forecasting mechanism discussed in this chapter can be used to support
outlier detection. Building on the previous section, this forecaster is
implemented using the neural network time-series forecaster with a hidden
layer:
public class NeuralNetworkForecaster implements
Forecaster {
NeuralNetwork nn;
int n;
double [] x;
double [] Y = new double [1];
public NeuralNetworkForecaster(NeuralNetwork nn) {
this .nn = nn;
}
public NeuralNetworkForecaster( int n) {
this (NeuralNetwork. build ().inputs(n).layer(n).layer(1));