Information Technology Reference
In-Depth Information
and cannot be automated. Point 18 can be partially automated, as analyzing the
results is always a human activity but error reporting can be automated fully.
9.1.2
Financial and Execution Time Impacts
Still it cannot be denied that automation of test case execution yields benefits. For a
large packaged enterprise software application, a suite of regression test cases could
easily run in more than several thousand test cases. Suppose that, on average, each
test case requires an execution time of 2 minutes; then the total time required for
running all regression test cases could be more than 4000 minutes (66 hours). That
means it will take more than eight working days if one person is assigned to run these
test cases manually. To reduce this execution cycle, the software vendor or his service
provider partner can assign eight people to complete the regression test case execution
in 1 day. Now suppose the average salary and other benefits of employees are $2000
per week. So the software vendor spends more than $2000 to run the regression test
cases for release of his software application. In addition to this amount of money, the
software vendor is spending more than one extra day in the release cycle. Now sup-
pose that all test cases have been automated and it takes 10 seconds to run each test
case by the automation tool. Then the time required to execute all regression test cases
comes to more than 330 minutes (5 hours). In this case the software vendor not only
is able to shrink the release cycle but will also save more than $2000.
Now suppose the company employs automation engineers to create automation
scripts for these 2000-odd test cases. Suppose they take 15 minutes to automate
each test case. That means it will take 500 hours to automate all test cases. So an
automation engineer will take 63 days to complete his assignment. Again suppose
the regression test suite runs once in every quarter (company releases new versions
once every quarter). Over a period of 5 years, the regression test suite will run 20
times. As we have already seen, the automated execution saves 61 hours in each
cycle; the time saving over the 5-year period will be 1220 hours. At the same time,
effort saving in terms of money spent in salary of the manual tester for executing
regression test cases for the 5 years will be $40,000. Though the money savings is
not big enough, the time savings is definitely a big differentiator. Time to market is
such an important factor that any time saving in a new release cycle is a big plus.
So the secret of being successful in automating does not lie with blindly auto-
mating test cases without thinking about what activities are being automated.
Along with automation, you also need to think about how to make the manual
activities that cannot be automated more efficient.
9.1.3
Workload Factor
Let us consider a business scenario for a software application vendor who makes
large enterprise applications. In the first release of the application, it took a team of
five test engineers to verify and validate the product in 10 business days. In the next
