Information Technology Reference
In-Depth Information
hardware. This section introduces the basic idea of parallel computing and its main
ingredients.
10.2.1
Inspirations from Parallel Processing in Real Life
In everyday life, it is very common to see several people join forces to work on
something together. Let us look at a scenario where two workers are set to paint
a wall. Of course, each worker should first have a brush and a can of paint. Then,
the two workers need to negotiate which area of the wall is to be painted by whom.
When this work division is settled, each painter can start painting his or her assigned
area, without having to interact with the other. The only assumption is that the two
painters do not fight with each other when painting the boundary zone. In case more
workers are available, the situation is similar except that the wall needs to be divided
into more and smaller areas.
Compared with letting one worker paint the wall, employing several workers will
normally speed up the project of wall painting. It has to be assumed, though, that
the work division is reasonable, i.e., all the workers are able to finish their assigned
areas using roughly the same amount of time. Otherwise the entire project of wall
painting is not done until the slowest worker finishes his part. A second observation
is that the initial negotiation of work division, which is not necessary for the single-
worker case, should not cost too much time overhead. A third observation is that you
cannot assign too many workers to paint a wall. That is, each worker should have
a sufficiently large area to paint, without frequently coming over to others' areas
and thereby slowing down each other. All in all, a speedup of the work for this very
simple case is possible to achieve, but requires a large enough wall plus caution with
respect to the work division.
There is, however, another important factor that is not addressed in the preced-
ing example: Collaboration between the workers is often needed while the work is
being carried out. Let us consider the example of bricklaying. Suppose a number of
masons are hired to set up a brick wall. A possible means of work division is to let
each mason have a vertical strip, inside of which she can lay the bricks layer-wise
from bottom to top. For reasons of structural strength, the masons should preferably
always work on the same layer of bricks. In addition, the positioning of bricks by
neighboring masons should match up, without leaving large gaps between sections
of bricks. The masons therefore need to collaborate, i.e., adopt a synchronized pace
and exchange ideas frequently.
10.2.2
From Serial Computing to Parallel Computing
Not unlike the above cases of parallel processing in real life, parallel computing is
only meaningful when an entire computational problem can be somehow divided
