Warehouse-Scale Computers to Exploit Request-Level and Data-Level Parallelism - Computer Architecture: A Quantitative Approach - page 482

same rack, or (3) memcached servers on the same racks and storage servers collocated
on separate racks.
6.20 [5/5/10/10/Discussion/Discussion] <6.3, 6.6> Datacenter Networking : Map-Reduce and
WSC are a powerful combination to tackle large-scale data processing; for example, Google
in 2008 sorted one petabyte (1 PB) of records in a litle more than 6 hours using 4000 servers
and 48,000 hard drives.
a. [5] Derive disk bandwidth from Figure 6.1 and associated text. How many seconds
does it take to read the data into main memory and write the sorted results back?
b. [5] Assuming each server has two 1 Gb/sec Ethernet network interface cards (NICs)
and the WSC switch infrastructure is oversubscribed by a factor of 4, how many
seconds does it take to shuffle the entire dataset across 4000 servers?
c. [10] Assuming network transfer is the performance botleneck for petabyte sort, can
you estimate what oversubscription ratio Google has in their datacenter?
d. [10] Now let's examine the benefits of having 10 Gb/sec Ethernet without oversub-
scription—for example, using a 48-port 10 Gb/sec Ethernet (as used by the 2010 Indy
sort benchmark winner TritonSort). How long does it take to shuffle the 1 PB of data?
e. [Discussion] Compare the two approaches here: (1) the massively scale-out approach
with high network oversubscription ratio, and (2) a relatively small-scale system with
a high-bandwidth network. What are their potential botlenecks? What are their ad-
vantages and disadvantages, in terms of scalability and TCO?f.
f. [Discussion] Sort and many important scientific computing workloads are communic-
ation heavy, while many other workloads are not. List three example workloads that
do not benefit from high-speed networking. What EC2 instances would you recom-
mend to use for these two classes of workloads?
6.21 [10/25/Discussion] <6.4, 6.6> Because of the massive scale of WSCs, it is very important
to properly allocate network resources based on the workloads that are expected to be run.
Diferent allocations can have significant impacts on both the performance and total cost of
ownership.
a. [10] Using the numbers in the spreadsheet detailed in Figure 6.13 , what is the oversub-
scription ratio at each access-layer switch? What is the impact on TCO if the oversub-
scription ratio is cut in half? What if it is doubled?
b. [25] Reducing the oversubscription ratio can potentially improve the performance if
a workload is network-limited. Assume a MapReduce job that uses 120 servers and
reads 5 TB of data. Assume the same ratio of read/intermediate/output data as in Fig-
ure 6.2 , Sep-09, and use Figure 6.6 to define the bandwidths of the memory hierarchy.
For data reading, assume that 50% of data is read from remote disks; of that, 80% is
read from within the rack and 20% is read from within the array. For intermediate
data and output data, assume that 30% of the data uses remote disks; of that, 90% is
within the rack and 10% is within the array. What is the overall performance improve-
ment when reducing the oversubscription ratio by half? What is the performance if it
is doubled? Calculate the TCO in each case.
c. [Discussion] We are seeing the trend to more cores per system. We are also seeing the
increasing adoption of optical communication (with potentially higher bandwidth and
improved energy efficiency). How do you think these and other emerging technology
trends will affect the design of future WSCs?