Storage Administration - Oracle Exadata Recipes: A Problem-Solution Approach - page 247

Let's walk through how Oracle allocates an extent with normal redundancy ASM disk groups:

• First, the primary extent is allocated on a primary disk (in the disk group).

• Next, ASM creates the second extent/copy on a different failure group from the primary copy.

• Each copy is on a disk in a different failure group, so loss of all disks in a failure group does not

result in data loss.

A failure group is simply a subset of disks within a disk group. Failure groups always exist, whether they're
manually created or not.
On Exadata, a failure group is created on each storage server. In the following examples, we're using an Exadata
Quarter Rack, with three storage servers as indicated by the three InfiniBand IP addresses in the disk strings, each
with 12 physical disks:

SQL> select a.name,b.path,b.state,a.type,b.failgroup
from v$asm_diskgroup a, v$asm_disk b
where a.group_number=b.group_number
and a.name like '%DATA%'
order by 2,1
/

Disk Group Disk State Type Failgroup
---------- ----------------------------------- ---------- --------- ----------------
DATA_CM01 o/192.168.10.3/DATA_CD_00_cm01cel01 NORMAL NORMAL CM01CEL01
DATA_CM01 o/192.168.10.3/DATA_CD_01_cm01cel01 NORMAL NORMAL CM01CEL01
DATA_CM01 o/192.168.10.3/DATA_CD_02_cm01cel01 NORMAL NORMAL CM01CEL01
... lines omitted for brevity

36 rows selected.

SQL>

The ASM disk string is of the form o/ <InfiniBand IP of the storage cell> / <Grid Disk name> . As you can
see, 36 disks comprise the DATA_CM01 ASM disk group, and each disk has its own failure group assigned. You can also
see that the redundancy type is normal, which means we've got two-way mirroring in place.
The failure group is assigned at the disk level. Extents are allocated and “striped” in chunks defined by the
ASM allocation unit size, or au_size , as discussed in the How It Works section of Recipe 9-2. If, for example, you are
formatting 64 MB worth of extents and a uniform extent size of 64 KB, Oracle allocates extents in the following manner:

• The primary extents for the first 64 KB extents are written to

o/192.168.10.3/DATA_CD_00_cm01cel01

CM01CEL01 , Oracle will choose a different failure group
for the mirrored extents—in this case, it'll take a disk from either CM01CEL02 or CM01CEL03

• Since the failure group for this disk is

to o/192.168.10.5/DATA_CD_00_cm01cel03 • The next set of primary extents are written

CM01CEL01 or CM01CEL02 disks • The mirrored extents for this next round are distributed to

o/192.168.10.4/DATA_CD_00_cm01cel02

• The next primary set of 64 extents will be written to

CM01CEL01 and CM01CEL03 • The mirrored extents for these will distributed between

If you expand this example and consider high redundancy ASM disk groups, then steps in the second, fourth, and
sixth bullets will write mirrored extents to disks in both the remaining failure groups.