Big Data Generation and Acquisition - Big Data: Related Technologies, Challenges and Future Prospects - page 27

Intra-DCN transmissions are the data communication flows within data centers.
Intra-DCN transmissions depend on the communication mechanism within the
data center (i.e., on physical connection plates, chips, internal memories of data
servers, network architectures of data centers, and communication protocols).
A data center consists of multiple integrated server racks interconnected with
its internal connection networks. Nowadays, the internal connection networks of
most data centers are fat-tree, two-layer or three-layer structures based on multi-
commodity network flows [ 23 , 26 ]. In the two-layer topological structure, the racks
are connected by 1 Gbps top rack switches (TOR) and then such top rack switches
are connected with 10 Gbps aggregation switches in the topological structure. The
three-layer topological structure is a structure augmented with one layer on the top
of the two-layer topological structure and such layer is constituted by 10 or 100 Gbps
core switches to connect aggregation switches in the topological structure. There
are also other topological structures which aim to improve the data center networks
[ 27 - 30 ].
Because of the inadequacy of electronic packet switches, it is difficult to increase
communication bandwidths while keeps energy consumption is low. Over the years,
due to the huge success achieved by optical technologies, the optical interconnection
among the networks in data centers has drawn great interest. Optical interconnection
is a high-throughput, low-delay, and low-energy-consumption solution. At present,
optical technologies are only used for point-to-point links in data centers. Such
optical links provide connection for the switches using the low-cost multi-mode
fiber (MMF) with 10 Gbps data rate. Optical interconnection (switching in the
optical domain) of networks in data centers is a feasible solution, which can provide
Tbps-level transmission bandwidth with low energy consumption.
Recently, many optical interconnection plans are proposed for data center
networks [ 31 ]. Some plans add optical paths to upgrade the existing networks, and
other plans completely replace the current switches [ 31 - 36 ]. As a strengthening
technology, Zhou et al. in [ 37 ] adopt wireless links in the 60 GHz frequency band to
strengthen wired links. Network virtualization should also be considered to improve
the efficiency and utilization of data center networks.

3.2.3 Data Pre-processing

Because of the wide variety of data sources, the collected datasets vary with respect
to noise, redundancy, and consistency, etc., and it is undoubtedly a waste to store
meaningless data. In addition, some analytical methods have stringent requirements
on data quality. Therefore, data should be pre-processed under many circumstances
to integrate the data from different sources, so as to enable effective data analysis.
Pre-processing data not only reduces storage expense, but also improves analysis
accuracy. Some relational data pre-processing techniques are discussed in the
following.