Testing for Scalability - Expert Oracle RAC Performance Diagnostics and Tuning - page 98

INST_ID STATUS Last Calibration Time
-------- ------------- ------------------------------
1 READY 10-FEB-13 11.50.38.279 AM
2 READY 10-FEB-13 11.50.38.279 AM
3 READY 10-FEB-13 11.50.38.279 AM
4 READY 10-FEB-13 11.50.38.279 AM
8 rows selected.
Running Test 3 again, there was no disk level I/O contention; however, after 30 minutes into the test, high CPU
utilization and network issues were visible.
Latency numbers from the disks where normal and acceptable. However, the EM grid control, iptraf, and
interactive verification against V$SYSSTAT indicated high latency on the interconnect.
INST_ID Avg CR Blk Recv Time(ms)
-------- ------------------------
1 6.7
2 10.5
3 12.3
4 12.4
It was also observed that the interconnect had several packet losses and errors on checking the blocks lost
statistics from V$SYSSTAT and O/S level statistics using netstat and ifconfig commands.
Checking the interconnect configuration indicated that jumbo frames were not configured. Datawarehouse
workloads data is transferred in larger packet sizes, much larger than the database block size of 8k. Jumbo frames
will help increase the transfer unit size from 1,500 to 9,000. This means the interconnect buffer sizes would be more
comfortable for an Oracle datablock of 8k or higher.
After considerable deliberation and convincing the network team that jumbo frames are used only in a private
network between the instances in the cluster and the fact that jumbo frames should not interfere with regular network
traffic outside the cluster, jumbo frames were enabled.
MOre OF a MYth FOr raC CONFIGUratION
the reasons why institute of electrical and electronics engineers (ieee 2 ) has not certified that jumbo frames
cannot be related to the configuration of jumbo frames in a closed raC environment. in a raC environment,
jumbo frame implementation is in a private network closed configuration. other networks should not be affected
by this configuration. in fact, if it does, there is potentially an issue in the way it's configured.
oracle supports and recommends the use of jumbo frames as a best practice for the private interconnect in a raC
configuration.
Step 6
After several iterations and testing of the cluster with different scalability factors, it was determined that the following
test (Table 4-4 ) had the right scalability numbers that the cluster could handle. The tests showed response times above
average and acceptable.
2 See Ethernet Alliance ( www.ethernetalliance.org ) publication Ethernet Jumbo Frames , Version 0.1, November 12, 2009.