Web Testing for Reliability Improvement - Advances in Computers - page 217

T ABLE VI
R ELIABILITY M ODELING R ESULTS FOR SMU/SEAS

Time/workload
measurement

Model parameters & estimates Reliability

growth ρ

N b λ 0 λ T

1 . 76 × 10 − 10 6 . 45 × 10 − 7 1 . 63 × 10 − 7 bytes 3674 0 . 748

1 . 38 × 10 − 6

hits 4213 0.00583 0.00203 0 . 632

1 . 42 × 10 − 5

sessions 4750 0.0675 0.0284 0 . 579

1 . 60 × 10 − 5

users 4691 0.0752 0.0311 0 . 587

alent to 26 days of operation could have reduced the failure rate to between slightly
less than half (1 − 57 . 9%) and about one quarter (1 − 74 . 8%) of the initial failure
rate. Other SRGMs we tried also yield similar results: A significant reliability im-
provement potential exists if we can capture the workload and usage patterns in log
files and use them to guide software testing and defect fixing.
We also repeated the assessment of reliability growth potential for the KDE web
site. However, when we extracted the unique failures (unique 404 errors), we no-
ticed an anomaly at the 24th day, which was associated with more than 10 times
the maximal daily unique errors for all the previous days. Further investigation re-
vealed that this is related to a planned beta release of the KDE product, when the
web contents were drastically changed and many new faults were injected. Since our
reliability growth evaluation is for stable situations where few or none new faults are
injected, as is the assumption for all the software reliability growth models [27,34] ,
we restricted our data to the first 22 days in this analysis.
Figure 12 plots the reliability growth evaluation we carried out for the KDE data.
Among the five workload measures we used, bytes, hits, users, s1 and s2 sessions,
all produced almost identical results in the reliability growth visualization, when
we plotted relative cumulative unique errors against relative cumulative workload,
similar to what we did in Fig. 11 . The comparative visualization is omitted here
because all the relative reliability growth curves would closely resemble the actual
curve represented by the actual data points in Fig. 12 . A visual inspection of Fig. 12
also revealed more degrees of reliability growth, or more bending of the data trend
and fitted curve, than that in Fig. 11 . Reliability growth potential as captured by ρ
for the KDE web site ranged from 86.7 to 88.9% (with the model in Fig. 12 gave us
ρ = 87 . 1%). In other words, effective web testing and defect fixing equivalent to 22
days of operation could have reduced the failure rate to about 11 to 13% (calculated
by 1 − ρ ) of the initial failure rate; or, equivalently, almost all the original problems
could have been fixed.