Large-Scale Co-Phylogenetic Analysis on the Grid - Cloud, Grid and High Performance Computing: Emerging Applications - page 234

The parameters -1, -2, -3 and -4 specify the
number of rows and columns in the association
matrix as well as the number of rows and columns
in the parasite and host matrices; -p represents
the number of permutations conducted by the
statistical test; -A, -B, and -C are used to read the
plain-text input files; -n specifies a run ID that is
appended to all output files (for details on the
AxParafit program parameters please refer to the
AxParafit manual at http://icwww.epfl.
ch/~stamatak/). The dataset we used is the afore-
mentioned (Section 2) dataset for the study of
smut-fungi, that was used to demonstrate perfor-
mance of the stand-alone AxParafit code by
Stamatakis et al. (2007). As already mentioned,
this dataset represents the largest real-world co-
phylogenetic study conducted to date. While the
sequential execution time for this dataset still
appears to be acceptable, such studies were previ-
ously not feasible with Parafit which is between
1-2 orders of magnitude slower than AxParafit.
Since the host-parasite association list contains
nz =2,362 entries, 2,362 individual tests need to
be performed. The execution of AxParafit to
compute global congruence of the trees returned
an estimated run time of 3 seconds per job, i.e.,
an overall expected run time of almost two hours
(2,362 x 3 seconds). The main goal of the first
test is therefore to minimize the expected response
time. We also executed the full test, as specified
above, on a single machine and observed that the
estimated run time of about 2 hours (7,000 seconds)
is almost identical to the measured run time (7,200
seconds). Therefore, we deduce that the run time
prediction mechanism is sufficiently accurate for
our application. In our experiments, we varied
the number of tasks (in the range between 60 and

162) as well as the number of parallel Grid jobs
(in the range between 24 to 124) to experimen-
tally determine the minimal response time. How-
ever, because of varying response times of the
Grid (i.e. the various Computing Elements and
their job queues etc.) it was not possible to deter-
mine an optimal number of Grid jobs and tasks.
Finally, in the experiment we used 124 Grid jobs
and 150 tasks which have been proposed by the
work distribution algorithm outlined in aforemen-
tioned section. The overall response time to
produce the final output was 11 minutes and 15
seconds (cf. Figure 4). Consequ e ntly, we observe
a clear runtime improvement with respect to a
single, sequential run. Note that the AxParafit.pl
program had to be adapted to allow for parallel
downloads of the individual results: originally,
results were downloaded sequentially, which
increased the overall response time by several
minutes. By overlapping communication with
computation, this problem was resolved.

Experiment with Synthetic Data

In another experiment, we used a larger ( synthetic )
test dataset that had been extracted from a larger
empirical dataset to test scalability of AxParafit
and compared the runtime of the Grid with the
infiniband cluster at the Technical University of
Munich equipped with 128 AMD Opteron 2.4 GHz
CPUs. In the association list, there were nz =2,048
non-zero entries (equivalent to 2,048 tasks) and
we used 100 permutations. The expected runtime
of a single task was 568 seconds, i.e., about 10
minutes. As a result, the expected sequential re-
sponse time to finish all 2,048 tasks is about 13.4
days. We used the wrapper as shown in Box 3.

Box 2.

AxParafit.pl -p 10 -1 413 -2 1400 -3 1390 -4 411 \
-A smuts010907.mat -B smuts010907_P.pco -C smuts010907_H.tra -n RUN_1