Information Technology Reference
In-Depth Information
Chapter 10
A Glimpse of Parallel Computing
It should not be too difficult to imagine that applications of scientific computing in
the real world can require huge amounts of computation. This can be due to a combi-
nation of advanced mathematical models, sophisticated numerical algorithms, and
high accuracy requirements. Such large-scale applications easily involve millions
(or more) of data entities, and thousands (or more) of time steps in the case of
time-dependent problems. All these will translate into a huge number of floating-
point arithmetic operations and enormous data structures on a computer. However,
a serial computer that has only one CPU will have trouble getting all these compu-
tations done quickly enough and/or fitting needed data into its memory. The remedy
to this capacity problem is to use parallel computing, for which the present chapter
aims to provide a gentle introduction.
10.1
Motivations for Parallel Computing
Let us start with motivating the use of multiple-processor computers by looking at
the limitations of serial computers from the perspectives of computing speed and
memory size.
10.1.1
From the Perspective of Speed
The development of computing hardware was well predicted by the famous Moore's
law [22], which says that the number of transistors that can be put in an integrated
circuit grows exponentially at a roughly fixed rate. Consequently, the speed of a
top serial computer, in terms of floating-point operations per second (FLOPS), kept
growing exponentially. This trend held from the 1950s until the beginning of the
twenty-first century, but it then showed signs of flattening out. A future serial com-
puter that can solve a problem that is too large for today's serial computer is unlikely.
Combing the forces of many serial computers in some way, i.e., parallel computing,
thus becomes a reasonable approach.
Search WWH ::




Custom Search