Data-Flow Management in Streaming Analysis - Real-Time Analytics - page 109

that writes the same data to all channels. The other built-in option is the
multiplexing selector. This selector uses information from the data element
to determine the destination channel of each piece of data arriving at the
source.
Figure 4.3
Sinks remove data from the channels and relay them to either a final output,
such as HDFS, or on to another agent. Unlike sources, sinks are each
attached to a single channel. Sending the output from multiple sinks to a
single agent is used to perform de-multiplexing.
Sinks can also be coordinated to some extent using sink processors. There
are three built-in sink processors: the default processor, a load-balancing
processor, and a failover processor. These are described in more detail later
in this chapter.
Configuring the Agent
Agents are configured, much like Kafka, using a properties file. It defines
both the arrangement of sources, channels, and sinks as well as the
configuration of the individual elements. The arrangement is the same for
all combinations, whereas the individual configurations depend on the type
of each of the elements. The configuration begins with the definition of
sources and sinks for each agent running on the machine, with the following
basic form:
<agent name>.sources= <source1> ... <sourceN>
<agent name>.channels= <channel1> ... <channelN>
<agent name>.sinks= <sink1> ... <sinkN>
The <agent name> and <source1> entries are replaced with unique
names that are used to identify these elements elsewhere in the properties