Information Technology Reference
In-Depth Information
This reduction in time savings is crucial. In product development cycles, prod-
uct vendors release major versions of their software on a quarterly basis. As is well
known, these software products have grown manifold over the last 10 years. Product
vendors have been adding new features in the products regularly. Whenever a new
version is to be released, all the existing features of the software have to be tested
to make sure that any modifications or extensions done in old features or changes
due to addition of new features do not affect the functionality of these features. To
accomplish this, regression testing is done with all releases. The suite of regression
test cases keeps increasing with addition/modification of features. So time required
to execute these test cases keep increasing with new releases. This leads to a delay
in the release of new versions. By automating these test cases, time required gets
reduced substantially over executing regression test cases manually. In some cases
this reduction is more than 90%.
In this case study, the customer benefited immensely by automation of regres-
sion test cases, as it allowed the customer to release new versions much faster. The
time savings on some applications came to more than 10 working days, which is
substantial for a quarterly release.
11.6.10.1
Key Highlights
Some of the highlights of the automation testing included 66% automation of test
case execution and management, mercury tools used for automation, traceability
matrix maintained, module-centric approach followed to minimize maintenance
effort, reusable components to be used across projects using keyword-driven frame-
work, common standards across all projects, trained and certified professionals,
and time and cost savings during regression testing.
11.6.10. 2
Sanity Testing
Sanity test cases on the production instance were executed on a daily basis. They
were automated completely by the service provider's team. They were also run auto-
matically at certain times of the day. Result of the executed sanity test was sent
automatically to support staff. On some applications, it took more than 1 hour to
run all sanity tests. Performance regression test cases as well as functional regres-
sion test cases were part of the sanity tests (see Figure 11.10).
Sanity tests were run nightly so that the application was available for testing
when no users were using the applications; as a result, testing activities did not
interfere with users' activities. If any problems were detected, they were communi-
cated to the support team, who acted swiftly to rectify them. This ensured that by
the time users started using the application at 9 a.m., the problems were fixed and
did not affect any users (see
Figure 11.11)
.
