Supporting Hardware - Expert Consolidation in Oracle Database 12c - page 74

Changes in the Hardware World
The world of hardware is changing at a very rapid pace. The core areas where most technological advance is visible
are processor architecture, the explosion of available memory, storage, and networking infrastructure. Combined
these changes present unique challenges not only to the operating system but also to the software running on it. Until
not too long ago it was unthinkable to have 160 CPU threads exposed to the operating system in the x86-64 world, and
scheduler algorithms had to be developed to deal with this large number. In addition to the explosion of the number
of CPU cores and threads you also have the possibility of vast amounts of memory at your disposal. Commercial
x86-64 servers can now address up to four terabytes. Non-uniform memory access (NUMA) has also become more
important on x86-64 platforms. Although the NUMA factor is most relevant in systems with four NUMA nodes and
more, understanding NUMA in Linux will soon become crucial when working with large servers.
Exadata was the first platform widely available running Oracle workloads and moving Remote Direct Memory
Access into the focus. Most of us associate Infiniband with RDMA, but there are other noteworthy applications, such
as iWARP and RoCEE (RDBMA over Converged Enhanced Ethernet), for example. Also, Infiniband is a lot more than
you might think: it is a whole series of protocols for various different use cases, ranging from carrier for classic IP
(“IP over IB”) to the low-latency Exadata protocol based on RDS or Reliable Datagram Sockets to transporting SCSI
(“SRP,” the SCSI RDMA Protocol).
Changes to storage are so fundamental that they deserve a whole section of their own. Classic fiber channel
arrays are getting more and more competition in form of PCI Express Flash Cards, as well as small-form factor
flash-arrays connected via Infiniband, allowing for ultra-low latency. New vendors are challenging the established
storage companies with new and interesting concepts that do not fit the model in which many vendors placed their
products over the last decade. This trend can only be beneficial for the whole industry. Even if the new storage
startups are quickly swallowed by the established players in the storage market their dynamics, products and ideas
will live on. No one can ignore the changes flash memory brought to the way enterprises store data anymore.
Thoughts About the Storage Backend
Oak Table member James Morle has written a great paper titled “Sane SAN 2010.” He has also predicated changes
to the way enterprise storage arrays are going to be built. With his statements he alluded to the flash revolution in
the data center. Over the last decade the use of NAND flash has become prolific, and that for a reason. Flash storage
can offer a lot more I/O operations per second in a single unit, while at the same time requiring a lot less space and
cooling. Given the right transport protocol, it can also boost performance massively. The pace of development of flash
memory outpaced the development for magnetic disk in the relatively short time it has existed. Advances in capacity
for magnetic disks have been made consistently in the past, but the performance data of the disks have not increased
at the same speed. Before starting a discussion about storage tiering and the flash revolution, a little bit of terminology
must be explained.
Bandwidth/throughput: these figures indicate how much data you can transmit between
your storage backend and the database host per unit of time. Most decision support
systems are highly bandwidth hungry.
Response time/latency: the quicker the I/O request can be satisfied, the better . Response
time is the time it takes from issuing an I/O request to its completion. Online transaction
processing systems are usually sensitive to changes in response times.
What are typical ball-park figures to expect from your storage array? In terms of latency you should expect 6-8
milliseconds for single block random reads and 1 or 2 milliseconds for small sequential writes such as log writes.
These figures are most likely not reads from physical disk, but rather are those reads affected by the caches in the
arrays. With the wide variety of existing storage solutions and the progress made on an almost monthly basis, it is next
to impossible to give a valid figure for expected throughput, which is why you do not find one here.