Database Reference
In-Depth Information
data, as well as some approaches to fitting the model in an “online” fashion
to the streaming data itself.
When it is possible to build models, those models need applications. The
two most popular types of applications for real-time data are monitoring
and optimization applications. It is no coincidence that they correspond
to the idea of identifying anomalous behaviors in the data and making
moment-to-moment decisions, respectively.
Monitoring involves the classical subject of real-time data. Collection of the
data to be monitored and visualization of those results has been covered at
lengthinpreviouschapters.Whatremainsistheidentificationofanomalous
events. The “Monitoring” section of this chapter discusses two types of
anomaly detection. The first type is outlier detection, when the system
enters an anomalous but ultimately transient state. The second type is
change detection where the system enters a fundamentally different state of
operation.
The last section of this chapter covers a hot topic in the Internet world,
optimization, where it is fairly easy to make changes and monitor outcomes.
Website optimization, in particular, is a popular subject with any number
of methods available for so-called A/B testing. In fact, most website traffic
monitoring software seems to have some sort of A/B testing framework
built-in. In this chapter, a specific technique called the multi-armed bandit
is used along with the modeling approaches from the first section to
implement optimization in a real-time environment.
Models for Real-Time Data
Anything that hopes to predict the behavior of a system must have an
underlying model that describes it. Ideally, this description is compact
relativetothedataitdescribes.Forexample,inNewtonianphysics,asimple
set of equations describing the actions of forces on a collection of objects is
sufficient to predict their motion over long periods of time.
With this concept in mind, a model can be broken into two parts. The
first is the behavior of the underlying system, which describes how various
components of the world interact to result in the observed behavior.
Unfortunately, it is very rare that all the variables that make up this model
can be observed, so it is usually not possible to completely determine this
model. It is, however, possible to construct a model that considers the
