Supporting Hardware - Expert Consolidation in Oracle Database 12c - page 78

infiniband is so much faster than Fiber Channel because of its incredibly low latencies, the high bandwidth, and
the zero-copy feature. Quad data rate (Qdr) has been available since 2008 and is in use in oracle's exadata.
it offers up to 10 gb/s per link, and exadata, just like most infiniband systems, has four-lane ports accumulating
to 40 gb/s, which is referred to as “Quad data rate”. the next evolution is already available today, called Fourteen
data rate infiniband. Fdr increases the link speed to 14 gb/s per link. it is to be expected that most iB ports will
be Qdr ports again, offering a total of 56 gb/s. a new encoding method also promises less overhead, but you are
well advised to use Fdr with pCie version 3 to make use of the enormous bandwidth on offer if you do not want
to bottleneck on the 2nd generation pCie express cards currently in use. infiniband is a new technology for most
of us, for which your organization needs to start acquiring hardware as well as operational know-how. this can be
a significant investment.

In an attempt to categorize flash-based storage irrespective of the way it is attached to the server and to put it
into perspective, you can refer to Hennessy and Patterson's Computer Architecture: A Quantitative Approach . There
you find a Memory Hierarchy ranging from super-fast access to registers within the processor to very slow access
to external storage such as tape. Sources vary in terms of access times, but the general dimensions are undisputed.
Consider the following shortened memory hierarchy in Table 3-1 :

Table 3-1. An Attempt to Define the Memory Hierarchy with a Select Few Examples

Storage tier Approximate latency Comment

Processor register Picosecond Processor registers store information retrieved from higher-tier

caches such as the processor L1 cache

Level 1 cache Less than 1 nanosecond Usually implemented as Static RAM (SRAM) as opposed to DRAM

Level 2 cache A few nanoseconds Can be slower if implemented off the chip but usually found on-die

DRAM ~30-50 nanoseconds,

depending on locality

Dynamic Random Access Memory is otherwise referred to as main
memory. A typical server would use DDR3 modules

Flash memory read microseconds Not taking the time into account to send the data to the host, i.e.,

no round-trip time. For that time you need to consider the Fiber
Channel roundtrip time or alternatively Infiniband roundtrips.

Flash memory write Usually longer than flash

memory read but still
within microseconds

Writes take longer on the physical layer; some solutions use caches
to mitigate the write penalty

15k Hard disk read Few milliseconds

Hennessy and Patterson also graphically plotted the access time for DRAM, which is used for “main memory”
in servers and hard disk in a logarithmic scale on the horizontal axis of a chart. The vertical axis denoted the cost per
gigabyte. As you can imagine, the access time for DRAM is very, very low; however, that comes at the expense of an
exorbitant cost per GB for DRAM. According to the graph a DRAM module provided approximately 50 nanoseconds
response time at the cost of roughly 100 dollars per GB. The same year the cost per GB magnetic storage was a lot
lower than 1 dollar per GB; however, the access time in nanoseconds could be around 8,000,000 nanoseconds, which
is the equivalent of 8 milliseconds. The costs are obviously relevant to when the graph has been created and does not
reflect current prices.