Supporting Hardware - Expert Consolidation in Oracle Database 12c - page 83

not in scope of this section, CPU 1, for example, can only directly talk to CPU 2. If it needs to communicate with CPU 3,
then it has to ask CPU 2 first, adding more latency. A typical dual-socket system is shown in Figure 3-2 :

Figure 3-2. Simplified schematic design of a two-socket system

Current x86-64 processors have multiple DDR3-memory channels—individual memory channels, however, are
not shown in this figure. The operating system has the task of masking the location of memory from its processes.
The goal is to allow applications not having to be NUMA-aware to execute on NUMA hardware. However, they should
be to take advantage of the local memory! Remember for the rest of the section that local memory access is faster,
hence preferred. If local memory cannot be used, then remote memory needs to be used instead. This can happen if
a process migrates (or has to migrate) from one CPU to another, in which case local and remote memory are reversed.
The previously mentioned point-to-point protocols allow the CPU to access remote memory. This is where the most
common NUMA-related problem lies. With every Oracle database it is important to have predictable performance.
There are a number of reports from users who have experimented with NUMA: some processes executed a lot faster
than without NUMA, but some others did not. The reason was eventually determined to be caused by access to
remote memory in a memory-sensitive application. This had the following consequences:

oracle-validated ” and “ oracle-rdbms-server-12cR1-preinstall ” RPMs disable
NUMA entirely by adding numa=off to the boot loader command line.

• Oracle's “

• Multiple notes on My Oracle Support do not openly discourage the use of NUMA but make it very

clear that enabling NUMA can cause performance changes. A change can be positive or negative;
you are also told to test the impact of enabling NUMA first, but that almost goes without saying.

So how can you make use of NUMA? First you have to enable it—check if your boot loader doesn't have the
numa=off line appended as shown here:

kernel /boot/vmlinuz-2.6.xxx ro root=... rhgb quiet numa=off

It might additionally be necessary to enable NUMA in the BIOS or EFI of your machine. After a reboot your server
is potentially NUMA aware. Linux uses the numactl package, among other software, to control NUMA. To list your
NUMA nodes in the server, you use the following command:

[oracle@server1 ~]$ numactl --hardware
available: 4 nodes (0-3)
node 0 cpus: 0 1 2 3 4 5