Biology Reference
In-Depth Information
accuracy is strikingly high (depending on methods, Pearson
of
87-98 %). It remains to be seen whether this will remain the case
over time—new aligners might be tempted to exploit HoT's idea in
their inference algorithms or parameter optimization procedures,
thus compromising its independence as a benchmarking criterion.
For instance, a trivial way of “gaming” the HoT score is to align
sequences with “centre-justification” (adding a gap character in the
middle of sequences of even-numbered length). Such obviously
flawed alignment procedure is nevertheless insensitive to joint
sequence reversals, consistently obtaining a perfect HoT score.
ρ
4
Structural Benchmarks
Benchmarks have also been developed starting from protein struc-
ture data. Structural benchmarks are by far the most widely adopted
type [
2
]. Most commonly these employ the superposition of known
protein structures as an independent means of alignment, to which
alignments derived from sequence analysis can then be compared
using the sum-of-pairs and true-column measures discussed earlier.
Structural benchmarks are naturally highly relevant when
sequence alignments are sought to identify structural concordance
among amino-acid residues. Yet they are also relevant to an evolu-
tionary interpretation of alignments. Indeed, the biological obser-
vation that forms the basis of using structure in the latter context is
that homologous proteins often retain structural similarity even
when sequence divergence is large [
29
]. Thus, at high levels of
divergence, a greater degree of confidence may be placed on align-
ments based on structural conservation than on sequence similarity.
If residues from different proteins can be shown to overlap in three-
dimensional space, it is likely (though not certain) that they are
homologous. An important advantage of structural benchmarks is
that they provide a truly independent, empirically derived standard
to test different alignment algorithms.
A number of structurally derived benchmark datasets exist.
One of the oldest is HOMSTRAD [
10
,
30
]. Although not origi-
nally intended for benchmarking, this dataset has been extensively
used to rate the quality of alignments. The first purpose-built,
large-scale structural benchmark was BAliBASE [
11
,
31
], which
was based on similarity of known protein structures. It is divided
into a number of datasets, each suited to test a different alignment
problem—for example, greater or lesser sequence diversity, the
presence of large insertions or extensions or the presence of
repeated elements. Each BAliBASE dataset was constructed by
accessing information in structural databases, and alignments
were verified by hand, at both the level of individual residues and
of overall secondary structure. Other purpose-built structural
benchmarks include SABMARK [
32
] and PREFAB [
33
], which
