Database Reference
In-Depth Information
agent1.sources.source1.spoolDir
. The source's channels are set with
agent1.sources.source1.channels
.
The sink is a
logger
sink for logging events to the console. It too must be connected to
a
file
channel, which means that events in the channel are persisted to disk for durabil-
ity. The system is illustrated in
Figure 14-1
.
Figure 14-1. Flume agent with a spooling directory source and a logger sink connected by a file channel
Before running the example, we need to create the spooling directory on the local filesys-
tem:
%
mkdir /tmp/spooldir
Then we can start the Flume agent using the
flume-ng
command:
%
flume-ng agent \
--conf-file spool-to-logger.properties \
--name agent1 \
--conf $FLUME_HOME/conf \
-Dflume.root.logger=INFO,console
The agent name must also be passed in with
--name
(since a Flume properties file can
define several agents, we have to say which one to run). The
--conf
flag tells Flume
where to find its general configuration, such as environment settings.
In a new terminal, create a file in the spooling directory. The spooling directory source ex-
pects files to be immutable. To prevent partially written files from being read by the
source, we write the full contents to a hidden file. Then, we do an atomic rename so the
source can read it:
[
91
]
