Database Reference
In-Depth Information
Host Interceptor
The Host interceptor attaches either the hostname of the Flume agent or
the IP address to the header metadata. The default header is called
host
,
but it can be changed using the
hostHeader
property. It also supports the
preserveExisting
property:
agent_1.sources.source-1.interceptors.i-1.type=host
agent_1.sources.source-1.interceptors.i-1.hostHeader=agent
agent_1.sources.source-1.interceptors.i-1.useIP=false
agent_1.sources.source-1.interceptors.i-1.preserveExisting=true
This interceptor is useful for maintaining audit trails during processing.
Static Interceptor
The Static interceptor is used to add a static header to all events. It takes two
properties,
key
and
value
, which are used to set the header:
agent_1.sources.source-1.interceptors.i-1.type=static
agent_1.sources.source-1.interceptors.i-1.key=agent
agent_1.sources.source-1.interceptors.i-1.value=ninetynine
Theprimaryusecaseforthisinterceptoristoidentifyeventsthathavetaken
different processing paths. By having agents add the path information to the
headers, the event can be later tracked through the system.
UUID Interceptor
This interceptor adds a globally unique identifier to each event it processes.
By default it sets the
id
header, but this can of course be changed:
agent_1.sources.source-1.interceptors.i-1.type=uuid
agent_1.sources.source-1.interceptors.i-1.headerName=id
agent_1.sources.source-1.interceptors.i-1.preserveExisting=true
As discussed at the beginning of the chapter, most of these systems
implement, at least once, delivery semantics that can occasionally lead to
duplicates. Depending on the application, it might be necessary to detect
and remove these duplicates. By placing this at the beginning of the Flume
topology, Events can be uniquely identified.
