Database Reference
In-Depth Information
x kafka_2.8.0-0.8.0/libs/log4j-1.2.15.jar
x kafka_2.8.0-0.8.0/libs/jopt-simple-3.2.jar
x kafka_2.8.0-0.8.0/libs/slf4j-simple-1.6.4.jar
x kafka_2.8.0-0.8.0/libs/scala-library.jar
x kafka_2.8.0-0.8.0/libs/zookeeper-3.3.4.jar
x kafka_2.8.0-0.8.0/bin/
[ More Output Omitted ]
$
Mostoperatingsystemconfigurationslimitthenumberofopenfilesallowed
by a process to a relatively small number, such as 1024. Due to a number
of factors, such as the number of topics, partitions, and retention time, it
is possible that Kafka will need to consume many more file handles than
this under normal operation. Increasing this limit depends on the operating
system, but on Linux it involves editing the
/etc/security/
limits.conf
file. For example, adding the following
nofile
(it stands for
“number of files” rather than “no files”) line increases the allowable open
files for the
kafka
user to 50,000:
kafka - nofile 50000
Kafka Prerequisites
Kafka's only external dependency is a ZooKeeper installation. In a
production environment, refer to the installation guide in the previous
chapter to get a ZooKeeper cluster running. For local development
environments, Kafka ships with a preconfigured ZooKeeper server that can
be used to host several brokers on a single machine.
To start this development version, Kafka includes a script
bin/
zookeeper-server-start.sh
and a configuration file
config/
zookeeper.properties
. You can use these as they are to start the server
in a terminal window:
$ cd kafka_2.8.0-0.8.0/
$ ./bin/zookeeper-server-start.sh config/
zookeeper.properties
INFO Reading configuration from:
config/zookeeper.properties
(org.apache.zookeeper.server.quorum.QuorumPeerConfig)
