Cryptography Reference
In-Depth Information
In each case the goal is to achieve maximal coverage with test cases that
completely check the behavior of the interface of the software. This covers two
aspects that are only loosely connected one with the other: A test driver that
runs through all the branches of a function can still leave errors undetected. On
the other hand, one can construct cases in which all the properties of a function
are tested, even though some branches of the function are not considered. The
quality of a test can thus be measured in at least two dimensions.
If to achieve a high degree of test coverage it does not suffice to establish
the test cases based simply on knowledge of the specification, which leads to
so-called
black box tests
, it is necessary to take into consideration the details
of the implementation in the construction of test cases, a modus operandi that
leads to so-called
white box tests
. An example where we have created test cases
for a special branch of a function based only on the specification is the division
algorithm on page 53: To test step 5 the special test data on page 65 were specified,
which have the effect that the associated code is executed. On the other hand, the
necessity of special test data for division by smaller divisors becomes clear only
when one considers that this process is passed off to a special part of the function
div_l()
. What is involved here is an implementation detail that cannot be derived
from the algorithm.
In practice, one usually ends up with a mixture of black box and white
box methods, which in [Dene] are aptly called
gray box tests
. However, it can
never be expected that one hundred percent coverage is to be achieved, as the
following considerations demonstrate: Let us assume that we are generating
prime numbers with the Miller-Rabin test with a large number of iterations (50,
say) and a corresponding low probability of error
4
−
50
10
−
30
(cf. Section
10.5) and then testing the prime numbers that are found with a further, definitive,
primality test. Since the flow of control leads to one or the other branch of the
program, depending on the outcome of this second test, we have no practically
relevant chance to reach the branch that is followed exclusively after a negative
test outcome. However, the probability that the doubtful branch will be executed
when the program is actually used is just as irrelevant, so that possibly one can
more easily live with doing without this aspect of the test than to alter the code
semantically in order to create the test possibility artificially. In practice, there are
thus always situations to be expected that require the abandonment of the goal of
one hundred percent test coverage, however that is measured.
The testing of the arithmetic functions in the FLINT/C package, which
is carried out primarily from a mathematical viewpoint, is quite a challenge.
How can we establish whether addition, multiplication, division, or even
exponentiation of large numbers produces the correct results? Pocket calculators
can generally compute only on an order of magnitude equivalent to that of the
standard arithmetic functions of the C compiler, and so both of these are of
limited value in testing.
≈
