Information Technology Reference
In-Depth Information
identical failures. This scenario represents the upper limit for the potential reliability
improvement if we attempt to fix operational problems on-line or if we attempt to
test the system and fix problems under simulated customer operational environment
using our integrated testing strategy described in Section
5
. This upper bound on
reliability growth may not be attainable under many circumstances because of the
large number of transient faults that usually take place whose origins are usually very
difficult to be identified and removed because of their dependency on the context.
Nevertheless, this upper limit gives us an idea about the potential reliability growth.
Should quantitative information become available about the faults that are hard to
fix, it can be used to fine tune the above limit to provide more accurate estimate of
reliability growth potential.
This limit on potential reliability improvement can be measured by the reliabil-
ity change (or
growth
) through the operational duration or testing process where
such defect fixing could take place. Under the web application environment, each
observed failure corresponds to a recorded type E error in the error log, and the ide-
alized defect fixing would imply no more observation of any duplicate type E errors.
In other words, failure arrivals under this hypothetical environment would resem-
ble the sequence of unique type E errors extracted from the error log, which can be
calculated by counting each type E error only once at its first appearance but not
subsequently.
In general, reliability growth can be visualized by the gradual leveling-off of the
cumulative failure arrival curve, because the flatter the part of the curve, the more
time it takes to observe the next failure. To visualize this, we plotted in
Fig. 11
cumulative unique failures versus different workload measurements we calculated
above. Relative scale is used to better compare the overall reliability growth trends.
The individual data points in the middle depict the failure arrivals indexed by cu-
mulative hits. The (top) solid line depicts failure arrivals indexed by the cumulative
bytes transferred. The (bottom) dashed line depicts failure arrivals indexed by the
cumulative number of users. The user session measurement resulted in almost iden-
tical curve shape as that for the number of users, thus was omitted to keep the
graph clean. As we can see from
Fig. 11
, there is an observable effect of relia-
bility growth for this data, with the tail-end flatter than the beginning for all three
curves.
Quantitative evaluation of reliability growth can be provided by software reliabil-
ity growth models (SRGMs) we described in Section
2
, which commonly assume
instantaneous defect fixing
[27,34]
. In this chapter, we use a single measure, the
purification level
ρ
[45]
to capture this reliability change:
λ
0
− λ
T
λ
0
λ
T
λ
0
ρ =
=
1
−
