Advanced Deployments - Implementing Splunk: Big Data Reporting and Development for Operational Intelligence - page 348

The lifecycle of a bucket
An index is made up of buckets , which go through a specific life cycle. Each bucket
contains events from a particular period of time.
As touched on in Chapter 10 , Configuring Splunk , the stages of this lifecycle are
hot , warm , cold , frozen , and thawed . The only practical difference between hot
and other buckets is that a hot bucket is being written to and has not necessarily
been optimized. These stages live in different places on disk and are controlled by
different settings in indexes.conf :
• homePath contains as many hot buckets as the integer value of
maxHotBuckets and as many warm buckets as the integer value of
maxWarmDBCount . When a hot bucket rolls, it becomes a warm
bucket. When there are too many warm buckets, the oldest warm
bucket becomes a cold bucket.
Do not set maxHotBuckets too low. If your data is not parsing perfectly,
dates that parse incorrectly will produce buckets with very large time
spans. As more buckets are created, these buckets will overlap, which
means all buckets will have to be queried every time, and performance
will suffer dramatically. A value of five or more is safe.
• coldPath contains cold buckets, which are warm buckets that have rolled
out of homePath once there are more warm buckets than the value of
maxWarmDBCount . If coldPath is on the same device, only a move is
required; otherwise, a copy is required.
• Once the values of frozenTimePeriodInSecs , maxTotalDataSizeMB ,
or maxVolumeDataSizeMB are reached, the oldest bucket will be frozen.
By default, frozen means deleted . You can change this behavior by
specifying either:
° coldToFrozenDir : This lets you specify a location to move buckets
once they have aged out. The index files will be deleted, and only
the compressed raw data will be kept. This essentially cuts disk
usage in half. This location is unmanaged, so it is up to you to
watch your disk usage.
° coldToFrozenScript : This lets you specify a script to perform some
action when the bucket is frozen. The script is handed the path to the
bucket about to be frozen.