Database Reference
In-Depth Information
and then produce a stream that can be consumed by downstream operators.
The analytical steps include various
operators
that perform specific actions
based on input from one or more streams. Finally, for every way
into
the con-
tinuous analytics platform, there are multiple ways
out
, and in Streams, these
outputs are called
sink adapters
. (Think of water flowing out of the tap and
into your kitchen sink.) Any operator can simultaneously be a source adapter,
standard operator, and sink adapter, but it is helpful to think of them as being
distinct. We cover all of these operators later in this chapter.
Streams applications generally fall into one of two types. The first keeps
up with the most current data no matter what happens, even if it means
dropping older data. For these applications,
now
is more important than pro-
cessing every bit of data. Examples of applications that need to keep up with
the most current data include detecting and responding to cyber attacks,
making buy-and-sell decisions on the stock market, or monitoring a person's
vital signs. For these applications, you need to make decisions based on the
most current data. Streams supports this type of application by delivering
scalability and high throughput, load-shedding operators to intelligently
shed data when necessary, and maintaining system-level high availability to
keep applications running.
The second type of application requires every bit of data to be processed,
no matter what. These applications typically have used database technology
in the past, but for reasons of efficiency, timeliness, or the inability to keep
up, a portion of the application is moved to Streams. One example of this
type of application is processing call detail records (CDRs) for the telecom-
munications industry. For compliance and business reasons, these CDRs
need to be de-duplicated, transformed, and stitched together without losing
a single record. By moving processing to Streams while the CDRs are still in
motion, significant efficiency gains are realized and new business opportuni-
ties arise. For these kinds of applications, the inherently high throughput
coupled with system-level high availability and application patterns make
Streams an excellent choice.
As with a web application server or a database, you can customize Streams
to deliver applications that solve business problems. Streams provides many
enterprise-class features, such as the ability to deploy applications across a
cluster of dozens or hundreds of servers to improve availability, scalability,
and performance (similar to Hadoop but for real-time applications). Streams
