Introduction to High-Performance Computing Using MPI - High-Throughput Image Reconstruction and Analysis - page 133

cator provides a context in which the communication is carried out. A communica-
tor identifies a group of MPI processes along with association of a unique rank to
each of the process in the group and a communication context. The default com-
municator MPI COMM WORLD contains all the MPI processes and assigns a unique
rank from 0 to P 1, where P is the number of processes in the job.

An MPI process can have different rank in the context of different commu-
nicators. Similarly, different communicators may have different sizes and contain
different groups of processes. The rank of a process in the context of a com-
municator and the size of the communicator can be queried using the functions
MPI Comm rank and MPI Comm size described earlier, with a suitable commu-
nicator argument.
Communicators can be dynamically created using MPI Comm dup which dupli-
cates a given communicator into another communicator, MPI Comm split which
partitions the group of processes in a given communicator into several disjoint
communicators, MPI Comm create which creates a new communicator using the
concept of MPI groups ,and MPI Intercomm create which creates an inter-
communicator to link two distinct communicators. The communicators can be
destroyed dynamically using MPI Comm free .
Messages sent or received in the context of a communicator cannot get mixed
with messages in the context of another communicator. Thus, communicators pro-
vide a modular way of matching point-to-point send and receive function calls, in
addition to the message tags. For collective operations, communicators provide a
modular way to hierarchically decompose MPI processes into smaller groups. Pro-
cesses in two disjoint communicators can independently make calls to collective
operations in the context of their own (disjoint) communicators, without inter-
fering with each other. These functions are very useful while writing MPI-based
reusable parallel libraries.

−

Process Topologies
MPI communicators may also be used to define virtual process topologies. These
topologies act as hints to the MPI subsystem indicating the expected communi-
cation pattern of the application. The MPI subsystem may use this information
to map the MPI processes to the hardware processors in a way that the specified
virtual process topology matches well with the underlying communication net-
work. This allows the application to optimally utilize the underlying network of
the system without losing its portability.
The function MPI Cart create creates a virtual topology of a general
d -dimensional torus with associated functions MPI Cart get , MPI Cart rank ,
and MPI Cart coords to get various attributes of the topology. The func-
tion MPI Cart sub is used to create a Cartesian subspace (similar to
MPI Comm split ) and the function MPI Cart shift is used to get infor-
mation needed to communicate with neighbors. Such topologies are very useful
in many applications. For example, 3D Cartesian topologies can be used for 3D
image filtering operations as demonstrated in Chapter 6.
For working with arbitrary graph topologies, functions MPI Graph create ,
MPI Graph neighbors count ,and MPI Graph neighbors are available.