Tuning the Cluster Interconnect - Expert Oracle RAC Performance Diagnostics and Tuning - page 445

Types of Interconnects
There are different types of interconnects available; some are proprietary, such as the LLT (low latency transport)
available from Veritas, or generic, such as the 10GigE and 1GigE interconnect. There are others that are high-speed,
top-of-line interconnects, such as the InfiniBand, or those that help keep latency low in long distance data transfer,
such as the blackfiber.
In this section, we discuss two of the common types of interconnects used in Oracle RAC implementations.

10 Gigabit Ethernet
Gigabit Ethernet has evolved from the original 10 Mbps Ethernet; 10 BASE-T; and the 100 Mbps Fast Ethernet
standards, 100BASE-TX and 100BASE-FX. The IEEE (Institute of Electrical and Electronics Engineers) and the
Gigabit Ethernet Alliance support a 10-Gigabit Ethernet (10GigE). 10GigE is the latest evolution in networking
options providing excellent high-speed communication between devices. Almost always, a GigE or 10GigE network is
sufficient for most types of workload computing, other technologies such as Infiniband (discussed later) may not be
required and could be an expensive solution.
Benefits of using Gigabit Ethernet over its predecessors include the following:

• 10GigE is 10 times faster than GigE and 100 times faster than 100Mbps Fast Ethernet

• Increased bandwidth for higher performance and elimination of bottlenecks

• Full-duplex capacity, allowing for increased bandwidth

• Full compatibility with the large installed base of Ethernet and Fast Ethernet nodes

• Transfer of large amounts of data quickly across networks

Oracle supports and recommends the use of user datagram protocol (UDP) for Linux/Unix environments and
TCP for Windows environments as the communication layer for the interconnect.
UDP is defined to make available a datagram mode of packet-switched computer communication in the
environment of an interconnected set of computer networks. The protocol is transaction oriented, and delivery
and duplicate protection are not guaranteed. 2 This protocol assumes that the Internet Protocol (IP) is used as the
underlying protocol.
TCP is a set of rules used along with the Internet Protocol (IP) to send data in the form of message units between
computers over the Internet. Whereas IP takes care of handling the actual delivery of the data, TCP takes care of
keeping track of the individual units of data (called packets) that a message is divided into for efficient routing
through the Internet.

InfiniBand Technology
The demands of the Internet and distributed computing are challenging the scalability, reliability, availability, and
performance of servers. InfiniBand™ architecture represents a new approach to I/O technology and is based on the
collective research, knowledge, and experience of the industry's leaders and computer vendors.
InfiniBand architecture specifies channels that are created by attaching host channel adapters (HCA) within a
server chassis to host channel adapters in other server chassis. This is done for high-performance IPC and to target
channel adapters connecting Infiniband-enabled servers to remote storage and communication networks through
InfiniBand switches. InfiniBand links transfer data at 2.5 Gbits/second, utilizing both copper wire and fiber optics for
transmission. It can carry any combination of I/O, network, and IPC messages.

2 Tierney, Brian L., Jason R. Lee, Dan Gunter, and Martin Stoufer. “Improving Distributed Application Performance Using TCP
Instrumentation,” May 2003. Lawrence Berkeley National Laboratory—Tom Dunigan, Oak Ridge National Laboratory.