Data-Flow Management in Streaming Analysis - Real-Time Analytics - page 132

To use it, simply set the channel.type to be jdbc . To define the path of
the database files, also set the sysprop.user.home property:
agent_1.channels.channel-1.type= jdbc
agent_1.channels.channel-1.sysprop.user.home= /home/
flume/db
There are other properties that can be set, but they do not generally need to
be changed.
Flume Interceptors
Interceptors are where Flume begins to blur the line between data motion
and data processing. The interceptor model allows Flume not only to modify
the metadata headers of an event, but also allows the Event to be filtered
according to those headers.
Interceptorsareattachedtoasourceandaredefinedandboundinmuchthe
same way as channels using the interceptors property:
agent_1.sources.source-1.interceptors= i-1 i-2
The order in which interceptors are defined in the configuration is the order
in which they are applied to the event. Because interceptors have the ability
to drop an event entirely, awareness of this ordering can be important if the
filtering interceptor depends on a header set by an earlier interceptor.
Flume provides a number of useful interceptors that can be used to decorate
events or provide useful auditing information during stream processing.
Some of the interceptors most commonly used are outlined in this section.
Timestamp Interceptor
The Timestamp interceptor adds the current timestamp as a metadata
header on the event with the key timestamp . There is an optional
preserveExisting property that controls whether or not the timestamp
is overwritten if it already exists on the event:
agent_1.sources.source-1.interceptors.i-1.type=timestamp
agent_1.sources.source-1.interceptors.i-1.preserveExisting=true