Storm in a Clustered Mode - Real-time Analytics with Storm and Cassandra - page 73

Cleaning up Zookeeper
We have seen how Zookeeper stores all its coordination data in the form of snapshots in the
path specified in the dataDir configuration. This requires periodic clean up or archival to
remove old snapshots so that we don't end up consuming the entire disk space. Here is a
small cleanup script that needs to be configured on all Zookeeper nodes:
numBackUps=3
dataDir=/usr/local/zookeeper/tmp
logDir=/mnt/my_logs/
echo `date`' Time to clean up StormZkTxn logs' >> $logDir/
cleanStormZk.out
java -cp /usr/local/zookeeper/zookeeper-3.4.5/zookeeper-
3.4.5.jar:/usr/local/zookeeper/zookeeper-3.4.5/lib/log4j-
1.2.15.jar:/usr/local/zookeeper/zookeeper-3.4.5/lib/
slf4j-api- 1.6.1.jar org.apache.zookeeper.server.PurgeTxnLog
$dataDir -n $numBackUps >> $logDir/cleanStormZk.out
Here we have the cleanup script as follows:
• numBackUps : Here we specify how many snapshots we want to retain after
cleanup; the minimum is three and the maximum can vary as per requirements.
• dataDir : Here we specify the path of the data directory where snapshots need to
be cleaned up.
• logDir : This is the path where the clean up script will store its logs.
• org.apache.zookeeper.server.PurgeTxnLog : This is the utility class
that purges all snapshots except the last three, as mentioned in numBackups .