Database Reference
In-Depth Information
1
Distributed Programming
for the Cloud
Models, Challenges,
and Analytics Engines
Mohammad Hammoud and Majd F. Sakr
CONTENTS
1.1 Introduction ...................................................................................................... 1
1.2 Taxonomy of Programs ..................................................................................... 2
1.3 Tasks and Jobs in Distributed Programs .......................................................... 4
1.4 Motivations for Distributed Programming ....................................................... 4
1.5 Models of Distributed Programs ...................................................................... 6
1.5.1 Distributed Systems and the Cloud ...................................................... 6
1.5.2 Traditional Programming Models and Distributed Analytics Engines .... 6
1.5.2.1 The Shared-Memory Programming Model ........................... 7
1.5.2.2 The Message-Passing Programming Model ........................ 10
1.5.3 Synchronous and Asynchronous Distributed Programs ..................... 12
1.5.4 Data Parallel and Graph Parallel Computations ................................. 14
1.5.5 Symmetrical and Asymmetrical Architectural Models ..................... 18
1.6 Main Challenges in Building Cloud Programs .............................................. 20
1.6.1 Heterogeneity ...................................................................................... 21
1.6.2 Scalability ........................................................................................... 22
1.6.3 Communication .................................................................................. 24
1.6.4 Synchronization .................................................................................. 26
1.6.5 Fault Tolerance .................................................................................... 27
1.6.6 Scheduling .......................................................................................... 31
1.7 Summary ........................................................................................................ 32
References ................................................................................................................ 34
1.1 INTRODUCTION
The effectiveness of cloud programs hinges on the manner in which they are
designed, implemented, and executed. Designing and implementing programs for
the cloud requires several considerations. First, they involve specifying the under-
lying programming model, whether message passing or shared memory. Second,
they entail developing synchronous or asynchronous computation model. Third,
1
