How Oracle Works on Exadata - Oracle Exadata Recipes: A Problem-Solution Approach - page 58

Disk Group Disk State TYPE Failgroup
---------- ----------------------------------- ------- ------ ---------------
DATA_CM01 o/192.168.10.3/DATA_CD_00_cm01cel01 NORMAL NORMAL CM01CEL01
DATA_CM01 o/192.168.10.3/DATA_CD_01_cm01cel01 NORMAL NORMAL CM01CEL01
DATA_CM01 o/192.168.10.3/DATA_CD_02_cm01cel01 NORMAL NORMAL CM01CEL01
... ASM disks for CM01CEL01 omitted
DATA_CM01 o/192.168.10.4/DATA_CD_00_cm01cel02 NORMAL NORMAL CM01CEL02
DATA_CM01 o/192.168.10.4/DATA_CD_01_cm01cel02 NORMAL NORMAL CM01CEL02
DATA_CM01 o/192.168.10.4/DATA_CD_02_cm01cel02 NORMAL NORMAL CM01CEL02
... ASM disks for CM01CEL02 omitted
DATA_CM01 o/192.168.10.5/DATA_CD_00_cm01cel03 NORMAL NORMAL CM01CEL03
DATA_CM01 o/192.168.10.5/DATA_CD_01_cm01cel03 NORMAL NORMAL CM01CEL03
DATA_CM01 o/192.168.10.5/DATA_CD_02_cm01cel03 NORMAL NORMAL CM01CEL03
... ASM disks for CM01CEL03 omitted

36 rows selected.

SQL>

This output tells us that DATA_CM01 is built with 36 Exadata grid disks. And while this doesn't necessarily confirm
that we're using an Exadata X2-2 Quarter Rack since an administrator can always build ASM disk groups on a list of
any number of grid disks, in our case and in most customer scenarios, ASM disk groups are typically constructed
using grid disks from each cell disk on every storage cell.
If you look at the second column in the previous output, you'll notice a disk path string convention of
o/ <IP address> / <grid disk name> . When building ASM disk groups on Exadata, this means the following:
The leading

•

o/ instructs Oracle ASM to search for disk paths using Exadata InfiniBand I/O
protocol—from the compute node, you can find your available storage server IP addresses by
looking in cellip.ora :

[grid@cm01dbm01 ~]$ locate cellip.ora
/etc/oracle/cell/network-config/cellip.ora
/opt/oracle.SupportTools/onecommand/tmp/cellip.ora
[grid@cm01dbm01 ~]$ cat /etc/oracle/cell/network-config/cellip.ora
cell="192.168.10.3"
cell="192.168.10.4"
cell="192.168.10.5"
[grid@cm01dbm01 ~]$

• The IP address in the second stanza of the disk string represents the InfiniBand IP address of

the Exadata storage cell. We can see three IP addresses, 192.168.10.3 , 192.168.10.4 , and
192.168.10.5 ; each of these is the bonded InfiniBand address of an Exadata storage cell.
The grid disk name in the third section of the ASM disk string represents the Exadata grid •

disk name.

When we created these ASM disk groups, we specified normal redundancy. With Exadata, external redundancy is
not an option—you either need to use normal or high redundancy. With normal redundancy, each extent is mirrored
to a different cell, and with high redundancy, they are mirrored via ASM to two additional cells. Specifically, extents
are mirrored to partner disks in different failure groups.