Database Reference
In-Depth Information
For durability, a log writer's
sync()
method is called, which gets the
acknowledgement from the low-level ilesystem on each update. This method also
takes care of writing the WAL to the replication servers (from one datanode to
another). The log lush time can be set to as low as you want, or even be kept in sync
for every edit to ensure high durability but at the cost of performance.
To take care of the size of the write-ahead log ile, the
LogRoller
instance runs as a
background thread and takes care of rolling log iles at certain intervals (the default
is 60 minutes). Rolling of the log ile can also be controlled based on the size and
hbase.regionserver.logroll.multiplier
. It rotates the log ile when it becomes
90 percent of the block size, if set to
0.9
.
HFile (the real data storage ile)
HFile represents the real data storage ile. The iles contain a variable number of data
blocks and a ixed number of ile info blocks and trailer blocks. The index blocks
records the offsets of the data and meta blocks. Each data block contains a magic
header and a number of serialized KeyValue instances.
Data
Block
Data
Block
Data
Block
Meta
Block
Meta
Block
File
Info
Data
Index
Trailer
.....
.....
Key
Value
Key
Value
Key
Value
Magic
.....
