Tuning the Storage Subsystem - Expert Oracle RAC Performance Diagnostics and Tuning - page 355

Queue Depths
There is a parameter called queue depth on the HBA that controls how much data is allowed to be “in flight” on the
storage network from that card. Most HBA cards default to a queue depth of 30 or 32, which is ideal for a general
purpose server and prevents the SAN from getting busy.
Maximum queue depth describes the request quantity that is allowed to be in flight; it can significantly
influence the storage performance. Queue depths should be set based on workload characteristics and performance
requirements. While low values can affect I/O performance negatively, setting this value to very high number could
also lead to bad performance. Default values that are set by the vendor may not be sufficient in many cases. Analysis
and tuning to determine the optimal value of queue depths is important in order to obtain good performance from
the I/O subsystem. (Refer to discussion on Little's Law on queues in Chapter 2.)
The output that follows is the default queue depth for an EMC storage array:

[root@tstdb1 sg]# cat device_hdr
host chan id lun type opens qdepth busy online
[root@tstdb1 sg]# cat devices
0 0 0 0 0 1 32 0 1
0 0 0 1 0 1 32 0 1
0 0 0 2 0 1 32 0 1
0 0 0 3 0 1 32 0 1
0 0 0 4 0 1 32 0 1
0 0 0 5 0 1 32 0 1
1 0 0 0 0 1 32 0 1
0 0 0 6 0 1 32 0 1
1 0 0 1 0 1 32 0 1
1 0 0 2 0 1 32 0 1
0 0 0 7 0 1 32 0 1

■ Certain storage vendors (for example, netapps) use dynamic queue management to adjust the queue depth
sizes based on workload.

Note

As we discussed in the previous section, the number of controllers on the storage array should be proportional to
the number of servers and their combined HBAs. If this is not the case, there could be queuing problems and delayed
I/O activity to the database. The number of hosts an array can adequately support depends on the available queues
per physical storage port, the number of storage ports, and the array's available bandwidth.
The iostat utility on Linux and Unix OS servers can be used to determine the queue lengths of the various
devices, as follows:

iostat -d -x interval count

-d : Display the device utilization report (d = disk) •

-x : Display extended statistics including disk utilization •

interval : The time period in seconds between two samples. iostat 5 will give data at
each 5-second interval

•

count : The number of times the data is needed. iostat 5 will give data 5 times in
5-second intervals

•