Experiments and Results - Protein Homology Detection Through Alignment of Markov Random Fields - page 42

Table 4.9 Reference-dependent alignment precision on the large benchmark Set60K
HMMER (%) HHalign (%) MRFalign (%)
Family 64.3 65.4 68.0
Superfamily 40.5 41.3 44.9
Fold 4.7 8.0 12.3
Family (beta) 67.4 68.1 72.3
Superfamily (beta) 45.4 46.2 49.4
Fold (beta) 6.7 9.2 14.5
The structure alignments generated by Matt are used as reference alignments. See Table 4.8 for
more explanation

Table 4.10 Reference-dependent alignment precision on the large benchmark Set60K
HMMER (%) HHalign (%) MRFalign (%)
Family 68.4 69.2 71.4
Superfamily 43.2 44.3 48.7
Fold 5.4 8.2 14.5
Family (beta) 70.8 72.4 77.9
Superfamily (beta) 46.6 48.4 53.7
Fold (beta) 7.9 8.6 17.8
The structure alignments generated by DeepAlign are used as reference alignments. See Table 4.8
for more explanation

On the very large set Set60K, as shown in Table 4.6 , MRFalign outperforms the
other two at each SCOP classification level regardless of the reference alignments
used. At the family level, MRFalign outperforms HHalign and HMMER by

3

*

and 4 %, respectively. At the superfamily level, our method outperforms

*

HHalign and HMMER by 5 %, respectively. At the fold level, MRFalign

outperforms HHalign and HHMER by

4 and

* *

5 and 8 %, respectively.

* *

4.5 Success Rate of Homology Detection
and Fold Recognition

To evaluate the success rate of homology detection and fold recognition, we
employ three benchmarks SCOP20, SCOP40 and SCOP80 introduced in [ 4 ]. For
each protein sequence in one benchmark, we treat it as a query, align it to all the
other proteins in the same benchmark and then examine if the query is similar to
those with the best alignment scores or not. We also tested the performance of on
these benchmarks hmmscan [ 5 ], FFAS [ 7 ], HHsearch [ 6 ] and HHblits [ 8 ], all of
which are run with default options. The success rate is measured at the superfamily
and fold levels, respectively. When evaluating the success rate at the superfamily
(fold) level, we exclude those proteins similar to the query at least at the family