Database Reference
In-Depth Information
Internal view HRegionServer
Region
Region
Region Server
MemStore
MemStore
Block Cache
Block Cache
Region Server
ZooKeeper
HFile
HFile
HFile
HFile
HFile
HFile
HFile
HFile
Region Server
HFile
HFile
HFile
HFile
Master
HDFS
HDFS
HDFS
As shown in the preceding diagram, the
HRegionServer
instance (the region server)
contains the map of
HRegion
instances (regions) and also has an
HLog
instance that
represents the WAL. There is a single block cache instance at the region-server level,
which holds data from all the regions hosted on that region server.
A block cache instance is created at the time of the region server startup and it can have
an implementation of
LruBlockCache
,
SlabCache
, or
BucketCache
. The block cache
also supports multilevel caching; that is, a block cache might have irst-level cache,
L1, as
LruBlockCache
and second-level cache, L2, as
SlabCache
or
BucketCache
.
All these cache implementations have their own way of managing the memory; for
example,
LruBlockCache
is like a data structure and resides on the JVM heap whereas
the other two types of implementation also use memory outside of the JVM heap.
HLog (the write-ahead log - WAL)
In the case of writing the data, when the client calls
HTable.put(Put)
, the data is
irst written to the write-ahead log ile (which contains actual data and sequence
numbers together represented by the
HLogKey
class) and also written in MemStore.
Writing data directly into MemStrore can be dangerous as it is a volatile in-memory
buffer and always open to the risk of losing data in case of a server failure. Once
MemStore is full, the contents of the MemStore are lushed to the disk by creating
a new HFile on the HDFS.
