Information Technology Reference
In-Depth Information
changed to fix a failed white-box test, it may require the repetition of all
the black-box tests.
To ensure completeness of test cases between black and white, it is
a good idea to do some mutation testing. This is a method of determining
if a set of test data or test cases is useful, by deliberately introducing
various code changes (“mutants”) and retesting with the original test data
or cases to determine if the “mutants” can be detected. The idea behind
mutation testing is simple: given an appropriate set of inputs, if a test set
identifies the mutants generated by these inputs (i.e., if it is able to find
these small errors), it will likely be good at finding real faults.
Regression Testing
In software, as in other systems, unintended consequences arise out of
complex interactions between elements of the system. These unintended
consequences can lead to misbehavior or failure of the product. For
example, a new installation changes a configuration file that affects another
application. An upgrade makes the historical data unreadable. Because
unintended consequences do occur, regression testing is necessary even
for systems that are highly modularized.
Many developers and testers do not understand the true nature of
regression testing. It is a common mistake to test Version 1 of their software
on Day 1's production data, and then test Version 2 with the production
data from Day 2. This is not a true regressive test because between the
two tests, both the software and the data have changed.
Let us illustrate regression testing with an example. Suppose one has
completed a document, run spell-check on it, and corrected all the spelling
mistakes. One now adds two more sentences to the document. Should
one run the spell-checker again? Yes, of course: the two new sentences
may have introduced spelling mistakes. So one runs spell-checker again.
This is an example of regression testing.
There are a few issues with running the spell-checker on the document
again. The “mistakes” in the earlier pass that had been flagged and ignored
get pointed out again and must be explicitly ignored again. This is the
reason why most regression testing is skipped. The problem lies with the
fact that a mistake is in the eye of the beholder — what a spell-checker
identifies as a mistake is actually an unrecognized word. The lack of
recognition is based on the lack of inclusion in some baseline dictionary.
Any dictionary is incomplete even with regard to commonly used words,
and certainly for words and terms of local usage within a company or
project. To avoid the flagging of such “mistakes,” one can either keep
adding exceptions to the dictionary, or adding clutter, or branch off local
dictionaries, which may not be possible to do in all word processors.
