Hardware Reference
In-Depth Information
FIGURE 33.2 (See color insert): Wavelength division multiplexing.
all-optical switches, including Accipiter systems, Calient Technologies, and
Huawei. The optical industry is already changing and growing to meet the
needs of HPC.
One technical design detail that remains outstanding is if the optical
switches will be all optical or a combination of optical and electrical (also
known as O-E-O). Along with this new technology there has to be a lot of
innovation with integrating wavelength multiplexing into an enhanced small
form-factor pluggable (SFP+) or another similar-size form factor. This en-
ables many ports on the line card of a switch or a couple of ports on a chip,
motherboard, or NIC, which lowers the cost of the total solution.
The file system storage network has to be a cost-ecient design while con-
tinuing to meet the storage and archival needs of the platforms. This network
relies on streaming bandwidth performance to file systems and typically does
not have the messaging rate requirements of a cluster interconnect. The cur-
rent cost-competitive network for ports and overall bandwidth is IB, even if
used solely for TCP/IP communications across the IB network. IB lacks func-
tioning as a truly enterprise network, but serves as a dedicated secondary clus-
ter storage network quite well. The use of an external-to-the-cluster storage
network allows for easier access from multiple clusters to a shared file system.
Technologies such as LANL's Parallel Scalable Backbone [5] (see Chapter 6)
and Lustre's Fine Grained Routing [6] minimize the cross-sectional bandwidth
required for the storage network when designing a network with multiple core
switches.
33.3 Challenges and Solutions
As with any new technology there are risks. DOE is trying to manage these
risks through the Fast Forward and Design Forward programs. The exascale
