Tuning the Cluster Interconnect - Expert Oracle RAC Performance Diagnostics and Tuning - page 447

buffer-related parameters and default values required to configure Oracle Database 11g Release 2 RAC or higher on
Linux are the following:
net.core.rmem_default = 262144
net.core.wmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_max = 4194304
net.ipv4.tcp_rmem=4096 4194304 4194304
net.ipv4.tcp_wmem=98304 4194304 4194304
net.ipv4.tcp_window_scaling=1
All of the preceding network parameters have a lower limit configured as the default value and a max value.
The default value is the startup value: the minimum allocation per socket immediately at socket creation. The max
value determines how much memory each socket is allowed to dynamically consume. As a best practice, it is advised
to set the default at the recommended values just listed and set the max to the maximum value supported by the O/S.
This would help conserve memory when possible and would allow dynamic growth if additional memory is required
by the sockets.
tcp_wmem
This variable takes three different values, which hold information on how much TCP send buffer memory space each
TCP socket has to use. Every TCP socket is allocated this buffer space to use before the buffer is filled up. Each of the
three values is used under different conditions.
The first value in this variable tells the minimum TCP send buffer space available for a single TCP socket; the
second value tells the default buffer space allowed for a single TCP socket to use; and the third value tells the kernel
the maximum TCP send buffer space. The /proc/sys/net/core/wmem_max value overrides this value.
tcp_rmem
The tcp_rmem variable tells the kernel the minimum receive buffer for each TCP connection. This variable takes three
different values, just the same as the tcp_wmem variable. The first value is for the TCP connection; the second value
specifies the default receive buffer allocated for each TCP socket; and the third value specifies the maximum receive
buffer that can be allocated to a TCP socket.
This value overrides the /proc/sys/net/core/rmem_default value.
tcp_mem
The tcp_mem variable defines how the TCP stack should behave when it comes to memory usage. It consists of
three values, just as the tcp_wmem and tcp_rmem variables. The three values define the low, medium, and maximum
threshold values for the TCP sockets. If the maximum value is reached, TCP streams, and we see packets being
dropped until the system stabilizes and the threshold reaches the lower value again.
Device Queue Sizes
Similar to tuning the network buffer sizes, it is important to look into the size of the queue between the kernel network
subsystems and the driver for network interface card. Inappropriate sizing can cause loss of data due to buffer
overflows, which in turn cause retransmission consuming resources and delays in performance. There are two queues
to consider in this area: the txqueuelen , which is related to the transmit/send queue size, and the netdev_backlog ,
which determines the receive queue size. The txqueuelen setting can be checked using ifconfig command on Linux
and Unix systems.