Database Reference
In-Depth Information
example, a production server could give inadvertent access to a staging
server. If that staging server were to be accidentally misconfigured, it could
potentially inject inappropriate data into the system.
Although the
type
,
bind
, and
port
properties are required, the Avro
source also allows for some optional properties. The optional properties
allow for compression of the Avro data stream by setting the
compression.type
to
deflate
. The stream may also be encrypted via
SSL (Secure Socket Layer) by setting the
ssl
property to
true
and pointing
the
keystore
property to the appropriate Java KeyStore file. The
keystore-password
property specifies the password for the KeyStore
file itself. Finally, the type of KeyStore is specified by
keystore-type
,
defaulting to JKS.
Thrift Source
PriortothereleaseofFlumeandKafka,ThriftandScribewere(andstillare)
popular choices for collecting logs in new infrastructures. The Thrift source
allows Flume to integrate with existing Thrift/Scribe pipelines. After you set
the
type
to
thrift
, you must set the address binding and the port as well.
Like the Avro source, take care when setting the address binding to 0.0.0.0:
agent_1.source.source-1.type=thrift
agent_1.source.source-1.bind=0.0.0.0
agent_1.source.source-1.port=4141
Netcat Source
Even simpler than the Avro and Thrift sources is the Netcat Source. For
those unfamiliar with
netcat
, it is a utility found in most UNIX-like
operating systems that can write or read to a raw TCP socket in much the
same way that the
cat
command reads and writes to file descriptors.
The Netcat source, like the Thrift and Avro sources, binds to a specific
address and port after you have set the type to
netcat
:
agent_1.source.source-1.type=netcat
agent_1.source.source-1.bind=0.0.0.0
agent_1.source.source-1.port=4141
