Distributed Programming for the Cloud - Large Scale and Big Data: Processing and Management - page 21

challenges that a designer needs to pay careful attention to and address before devel-
oping a cloud program. We next discuss heterogeneity , scalability , communication ,
synchronization , fault-tolerance , and scheduling challenges exhibited in building
cloud programs.
1.6.1 h eterogeneity
The cloud datacenters are composed of various collections of components includ-
ing computers, networks, operating systems (OSs), libraries, and programming lan-
guages. In principle, if there is variety and difference in datacenter components, the
cloud is referred to as a heterogeneous cloud . Otherwise, the cloud is denoted as a
homogenous cloud . In practice, homogeneity does not always hold. This is mainly
due to two major reasons. First, cloud providers typically keep multiple generations
of IT resources purchased over different timeframes. Second, cloud providers are
increasingly applying the virtualization technology on their clouds for server con-
solidation, enhanced system utilization, and simplified management. Public clouds
are primarily virtualized datacenters. Even on private clouds, it is expected that vir-
tualized environments will become the norm [83]. Heterogeneity is a direct cause
of virtualized environments. For example, co-locating virtual machines (VMs) on
similar physical machines may cause heterogeneity. Specifically, if we suppose two
identical physical machines A and B , placing 1 VMover machine A and 10 VMs
over machine B will stress machine B way more than machine A , assuming all VMs
are identical and running the same programs. Having dissimilar VMs and diverse
demanding programs are even more probable on the cloud. An especially compel-
ling setting is Amazon EC2. Amazon EC2 offers 17 VM types [1] (as of March 4,
2013) for millions of users with different programs. Clearly, this creates even more
heterogeneity. In short, heterogeneity is already, and will continue to be, the norm
on the cloud.
Heterogeneity poses multiple challenges for running distributed programs on the
cloud. First, distributed programs must be designed in a way that masks the hetero-
geneity of the underlying hardware, networks, OSs, and the programming languages.
This is a necessity for distributed tasks to communicate, or otherwise, the whole
concept of distributed programs will not hold (recall that what defines distributed
programs is passing messages). To elaborate, messages exchanged between tasks
would usually contain primitive data types such as integers. Unfortunately, not all
computers store integers in the same order. In particular, some computers might
use the so-called big-endian order, in which the most significant byte comes first,
while others might use the so-called little-endian order, in which the most signifi-
cant byte comes last. The floating-point numbers can also differ across computer
architectures. Another issue is the set of codes used to represent characters. Some
systems use ASCII characters, while others use the Unicode standard. In a word,
distributed programs have to work out such heterogeneity so as to exist. The part
that can be incorporated in distributed programs to work out heterogeneity is com-
monly referred to as middleware . Fortunately, most middleware are implemented
over the Internet protocols, which themselves mask the differences in the underly-
ing networks. The Simple Object Access Protocol (SOAP) [16] is an example of a