Database Reference
In-Depth Information
Disks and Partitions
The first thing you are going to want to check is the amount of free disk space
on data partitions and the CommitLog partitions (assuming they are on separate
partitions). Remember that if you are using SizeTieredCompaction, you shouldn't
have the alert set for
WARNING
at 80% disk utilization and
CRITICAL
set at
90% disk utilization. The safer approach is to set the
WARNING
threshold to be
roughly 35% disk utilization and the
CRITICAL
threshold at 45% disk utilization.
SizeTieredCompaction is capable of taking up two times the size of the largest
SSTable on disk. And while it is unlikely that a single SSTable would be 50% of
the data on disk, it is better to be safe than sorry. Recovering from having too much
data on disk is extremely difficult.
This concept of monitoring partitions and drives is also important because of
JBoD support in Cassandra 1.2 and later. This means that Cassandra can have a
single data directory on multiple disks. You will need to know if one or more of
those disks are having an issue or require replacement. By monitoring the utiliza-
tion and health of all the disks in your system, you will know their state and wheth-
er they need replacing or maintenance.
Last, you want to ensure that the drive that contains the log files doesn't fill up.
Depending on your log settings, Cassandra has the potential to be very verbose in
the log files. If the log files become too large, they can prevent the rest of your
system from working if the drive(s) runs out of space.
Swap
Linux divides its physical memory into smaller chunks called pages. Swapping
is the process whereby a page of memory is copied from memory to a dedicated
space on the hard disk called swap space to free up that page of memory. Although
there are cases where it is OK, it is normally not recommended for systems to be
in a state where they are swapping memory. Typically, anything more than 5% to
10% of your swap space being used is cause for investigation.
On a Cassandra node, swapping is usually a bad sign, so you will want to mon-
itor the swap partition for usage of nearly any kind. Since you should be able to
hold the entire JVM's heap space in memory with at least a little room to spare
for the operating system, getting to the point of swapping out pages of memory
means it might be a little too late to recover. One of the reasons Cassandra is able
to function so well with regard to writes is the fact that many of the writes occur
to the memory-mapped MemTables. Having these MemTables swap to disk would
