Information Technology Reference
In-Depth Information
TABLE 8.2
Running Time of Each Part in Single Machine (Bold Indicates Longest)
Runtime (seconds)
Raw Data/
Measuring Time
(hours)
Filtering
Baseline
Wander
Detecting the
QRS Wave
Denoising
Others
Total Time
00007732/21.5
194.281000
7.829000
59.406000
179.515000
441.046000
00016412/13
117.110000
4.734000
11.734000
5.079000
138.657000
0039720/3
26.672000
1.078000
1.469000
2.656000
31.875000
01297217/12
104.000000
4.344000
6.500000
32.156000
147.015000
01334816/14.5
124.797000
5.234000
12.594000
27.094000
169.719000
client should be uploaded to MongoDB server by the remote user. To obtain
these input data by the demon running on the Hadoop platform, they should
first be downloaded from MongoDB locally and then uploaded to HDFS
for further analysis and processing. In this scenario, data are pulled from
MongoDB and processed within Hadoop via one or more Map-Reduce jobs.
Output from these Map-Reduce jobs can then be written back to MongoDB
for later querying and ad hoc analysis. Communication between client and
platform is implemented by RabbitMQ, which is a popular MQ middleware.
Assuming that the data contain 24-hour ECG signals, you might consider
designing three map functions, each for 8 hours of the data, and then com-
puting in parallel. Replications of processed data on HDFS are output files.
The whole procedure is shown in Figure 8.9.
Each split represents a segment of data in the filtering baseline wander
parallel programming. Each phase has key value pairs as input and output,
in which the key stands for the data fragmentation flag, while the value stands
for the ECG raw data or processed data (i.e., in the form of <
raw data >, and < >, respectively). Since the default
implementation of the interface InputFormat in Hadoop is TextInputFormat ,
24-hour
Raw Data
Input HDFS
sort
Output HDFS
8-hour ECG
Split()
Map
copy
merge
HDFS
replication
reduce
part 0
8-hour ECG
Split 1
Map
reduce
part 1
HDFS
replication
8-hour ECG
Split 2
Map
FIGURE 8.9
The Map-Reduce for filtering baseline wander.
 
Search WWH ::




Custom Search