Web Testing for Reliability Improvement - Advances in Computers - page 180

as a black-box that prevents us from seeing the contents inside [18] . Structural or
white-box testing focuses on the internal implementation, while viewing the object
to be tested as a white-box that allows us to see the contents inside [5] .
Most of the traditional testing techniques and testing sub-phases use some cov-
erage information as the stopping criteria [6,35] , with the implicit assumption that
higher coverage means higher quality or lower levels of defects. On the other hand,
product reliability goals can be used as a more objective criterion to stop testing. The
use of this criterion requires the testing to be performed under an environment that
resembles actual usage by target customers so that realistic reliability assessment can
be obtained, resulting in the so-called usage-based statistical testing [30,34] .
The steps in most systematic testing include [7,48] :
(1) information gathering, with the internal implementations or external functions
as the main information sources giving us white-box testing and black-box
testing, respectively;
(2) test model construction, with models based on checklists, partitions, equiv-
alence classes, and various forms of finite-state machines (FSMs) such as
control flow graphs and data dependency graphs;
(3) test case definition and sensitization, which determines the input to realize a
specific test;
(4) test execution and related problem identification; and
(5) analysis and followup, including making decisions such as what to test next
or when to stop.
For usage-based statistical testing that play a critical role to ensure product reliability,
the following particular steps are involved:
The information related to usage scenarios, patterns, and related usage frequen-
cies by target customers and users is collected.
The above information is analyzed and organized into some models—what we
call operational profiles (OPs)—for use in testing.
Testing is performed in accordance with the OPs.
Testing results is analyzed to assess product reliability, to provide feedback, and
to support follow-up actions.
Both the failure information and the related workload measurements provide us
with data input to various software reliability models that help us evaluate the current
reliability and reliability change over time [27,34,44] . Two basic types of software
reliability models are: input domain reliability models (IDRMs) and time domain
software reliability growth models (SRGMs). IDRMs can provide a snapshot of the
current product reliability. For example, if a total number of f failures are observed