Database Reference
In-Depth Information
based on certain criteria, gets flushed to a disk in immutable files called SSTable. The data
in commit logs gets purged after its corresponding data in MemTable gets flushed to SST-
able.
Also, there exists one single commit log per node server. Like any other logging mechan-
ism, the commit log is set to rolling after a certain size. The following figure shows the
commit log, MemTable, and SSTable in action:
Let's quickly go a bit deeper into the implementation. All the classes that deal with the
commit log management reside under the
org.apache.cassandra.db.commitlog
package. The commit log singleton is a
facade for all the operations. The implementations of
ICommitLogExecutorSer-
vice
are responsible for write commands to the commit log file. Then there is a
Com-
mitLogSegment
class. It manages a single commit log file, writes serialized write
(mutation) to the commit log, and it holds a very interesting property:
cfLastWrite
.
The
cfLastWrite
property is a map with a key as the column family name and value
as an integer that represents the position (offset) in the commit log file where the last
mutation for that column family is written. It can be thought of as a cursor; one cursor per
column family. When the MemTable of a column family is flushed, the segments contain-
ing those mutations are marked as clean (for that particular column family). And when a
new write arrives, it is marked dirty with offset at the latest mutation.
In events of failure (hardware crash, abrupt shutdown), this is how the commit log helps
the system to recover:
1. Each commit log segment is iterated in the ascending order of timestamp.
2. Lowest
ReplayPosition
(which is the offset in commit log that specifies the
point till which the data is already stored in SSTables) is chosen from the SSTable
metadata.
3. The log entry is replayed for a column family if the position of the log entry is
greater than the replay position in the latest SSTable metadata.
