Literature review - Reliability Assurance of Big Data in the Cloud - page 13

2.2 Data reliability assurance in software

Apart from research studies on reliability theories for storage devices, many efforts
for ensuring data reliability have also been made in the software aspect. In this sec-
tion, we summarize existing literature on providing data reliability assurance with
software-based approaches. Essentially, all the approaches in this literature achieve
the goal of data reliability assurance by adding redundancy to the data. In general,
these approaches can be categorized as two major types, which are data replication
and erasure coding. 1 Both of these approaches have been widely applied to existing
distributed storage systems, which form two storage schemas: replication-based data
storage schema and erasure coding-based data storage schema. These two storage
schemas have their own advantages and disadvantages and are useful for different sce-
narios. In this section, these two kinds of approaches with their corresponding storage
schemas are reviewed, respectively.

2.2.1 Replication for data reliability
Among all the existing approaches for adding data redundancy and supporting data re-
liability, data replication has been considered a dominant approach in current distrib-
uted data storage systems. Currently, distributed storage systems that leverage replica-
tion for providing data reliability include ThriftStore [46] , Farsite [47] , Total Recall
[48] , Google File System (GFS) [9] , Hadoop Distributed File System (HDFS) [8] ,
Amazon S3 [10] , Parallel Virtual File System (PVFS) [49] , Ceph Filesystem (Ceph
FS) [50] , FreeLoader [51] , and many others. Specifically, Total Recall uses replication
for small files and erasure coding for large files; Windows Azure Storage [52] uses
replication for “hot” data and erasure coding for older yet less used data to reduce the
storage cost. Therefore, in these two systems both storage schemes are used.
Data replication-related research studies have been conducted for many years, and
many approaches on this topic for data reliability related issues in distributed storage
systems have been proposed [46,53-59] . Review articles include a detailed survey on
reliability issues of grid systems [54] in which data replication research studies for
the reliability of grid systems are comprehensively reviewed and important issues and
reviews of data reliability research in grid environments are also identified. A series of
optimistic replication algorithms (also called “lazy” replication algorithms) has been
comprehensively surveyed [56] , which synchronizes changes in replicas in the back-
ground, discovers conflicts after they happen, and reaches agreement on the final con-
tents incrementally. In addition, key challenges of optimistic replication systems are
also identified, such as ordering operations, detecting and resolving conflicts, propagat-
ing changes efficiently, bounding replica divergence, and so forth.
For describing data reliability of replication-based systems, analytical data reliabil-
ity models have been proposed and comprehensively studied [4,19,55,57,60] . Among

1 In addition to these two categories, there do exist some hydrated storage systems that leverage both data
replication and erasure coding approaches.