Advanced Deployments - Implementing Splunk: Big Data Reporting and Development for Operational Intelligence - page 349

• thawedPath can contain buckets that have been restored. These buckets are
not managed by Splunk and are not included in All time searches. To search
these buckets, their time range must be included explicitly in your search.
I have never actually used this directory. Search http://splunk.com for
restore archived , for procedures.
Sizing an index
To determine how much disk space is needed for an index, use the
following formula:
(gigabytes per day) * .5 * (days of retention desired)
Likewise, to determine how many days you can store an index, the formula
is essentially:
(device size in gigabytes) / ( (gigabytes per day) * .5 )
The .5 represents a conservative compression ratio. The log data itself is usually
compressed to 10 percent of its original size. The index files necessary to speed
up search brings the size of a bucket closer to 50 percent of the original size,
though it is usually smaller than this.
If you plan to split your buckets across devices, the math gets more complicated
unless you use volumes. Without using volumes, the math is essentially as follows:
• homePath = (maxWarmDBCount + maxHotBuckets) * maxDataSize
• coldPath = maxTotalDataSizeMB - homePath
For example, say we are given these settings:
[myindex]
homePath = /splunkdata_home/myindex/db
coldPath = /splunkdata_cold/myindex/colddb
thawedPath = /splunkdata_cold/myindex/thaweddb
maxWarmDBCount = 50
maxHotBuckets = 6
maxDataSize = auto_high_volume #10GB on 64-bit systems
maxTotalDataSizeMB = 2000000
Filling in the preceding formula, we get these values:
homePath = (50 warm + 6 hot) * 10240 MB = 573440 MB
coldPath = 2000000 MB - homePath = 1426560 MB
If we use volumes, this gets simpler and we can simply set the volume sizes to our
available space and let Splunk do the math.