Information Technology Reference
In-Depth Information
because FSM-based testing can generally handle up to a few dozen states at most
[5,48]
.
Hierarchical FSMs can be used to alleviate the problem associated with the large
numbers of states and transitions by limiting transactions across boundaries of dif-
ferent FSMs in the hierarchy: In lower-level models, we generally assume a common
source and common sink as its interface with higher-level models. However, the in-
teractions may well be cutting through hierarchy boundaries in real systems. For
large web sites, the complete coverage of all these hierarchical FSMs would still be
impractical.
Another alternative to deal with state explosion problem of FSMs is to use selec-
tive testing based on Markov operational profiles (Markov OPs)
[30,56]
by focusing
on highly used states and transitions while omitting infrequently used ones or group-
ing them together. The combination of these hierarchical FSMs and Markov OPs led
us to unified Markov models (UMMs)
[20,49]
described in Section
5
. On the other
hand, loosely related collections of web pages can be more appropriately represented
and tested by using some simpler usage models based on a flat list of operations and
associated probabilities, such as Musa's operational profiles (OPs)
[33]
. The intro-
duction of statistical testing strategies based on such OPs or UMMs is not to replace
traditional testing techniques, but to use them selectively on important or frequently
used functions or components.
On the other hand, as a general rule, usage and problem distribution among dif-
ferent software components is highly uneven
[8]
, which is also demonstrated to be
true among different web contents
[25]
. Consequently, some kind of selective testing
is needed to focus on highly-used and problematic areas to ensure maximal web site
reliability improvement, such as through usage-based statistical testing we discuss in
the next section. These techniques can help us prioritize testing effort based on us-
age scenarios and frequencies for individual web resources and navigation patterns
to ensure the reliability of web-based applications.
5.
A Hierarchical Approach for Large-Scale Web Testing
We next describe an integrated strategy that combines several existing testing
techniques in a hierarchical framework for web testing
[48,50]
. We first outline the
overall framework and then describe its individual tiers in detail.
5 . 1 O v e r a l l F r a m e w o r k : A T h r e e - T i e r e d W e b Te s t i n g S t r a t e g y
Once we have decided to use usage-based statistical web testing, the immediate
question is then the choice between two commonly used usage models or operational
