Database Reference
In-Depth Information
On reads, Cassandra will check the memtable first to find the value. Memtables are implemented
by the
org.apache.cassandra.db.Memtable
class.
Hinted Handof
Consider the following scenario. A write request is sent to Cassandra, but the node where the
write properly belongs is not available due to network partition, hardware failure, or some other
reason. In order to ensure general availability of the ring in such a situation, Cassandra imple-
ments a feature called hintedhandof. You might think of a hintas a little post-it note that con-
tains the information from the write request. If the node where the write belongs has failed, the
Cassandra node that receives the write will create a hint, which is a small reminder that says, “I
have the write information that is intended for node B. I'm going to hang onto this write, and
I'll notice when node B comes back online; when it does, I'll send it the write request.” That is,
node A will “hand off” to node B the “hint” regarding the write.
This allows Cassandra to be always available for writes, and reduces the time that a failed node
will be inconsistent after it does come back online. We discussed consistency levels previously,
and you may recall that consistency level
ANY
, which was added in 0.6, means that a hinted han-
doff alone will count as sufficient toward the success of a write operation. That is, even if only a
hint was able to be recorded, the write still counts as successful.
Some concern about hinted handoffs has been voiced by members of the Cassandra community.
At first, it seems like a thoughtful and elegant design to ensure overall durability of the database,
and appears unproblematic because it is familiar from many distributed computing paradigms,
such as Java Message Service (JMS). In a durable guaranteed-delivery JMS queue, if a message
cannot be delivered to a receiver, JMS will wait for a given interval and then resend the request
until the message is received. But there is a practical problem with both guaranteed delivery in
JMS and Cassandra's hinted handoffs: if a node is offline for some time, the hints can build up
considerably on other nodes. Then, when the other nodes notice that the failed node has come
back online, they tend to flood that node with requests, just at the moment it is most vulnerable
(when it is struggling to come back into play after a failure).
In response to these concerns, it is now possible to disable hinted handoff entirely, or, as a less
extreme measure, reduce the priority of hinted handoff messages against new write requests.
NOTE
In Cassandra 0.6 and earlier,
HintedHandoffManager.sendMessage
would read an entire row into
memory, and then send the row back to the client in a single message. As of version 0.7, Cassandra will
now page within a single hinted row instead. This can improve performance against very wide rows.
