The Testing Environment - Software Testing: Testing Across the Entire Software Development Life Cycle - page 157

A less obvious downside of not controlling the testing environment is that
someone other than the testers may place rogue software in the testing environment
that affects the test runs unknowingly and undetected. The fi rst clue that there is rogue
software in the test environment arises when test execution results cannot be repeated.
Decisions to reject automated staging tools in favor of manual procedures also im-
pact the testing validity. The primary intent of manual staging is usually to circumvent
the staging rules and be able to easily introduce alternate versions of the software
components to be tested. Although this approach seems to offer more fl exibility than
staging tools on the surface, the fact is that circumventing the staging rules diminishes
the veracity of subsequent test results. Furthermore, circumventing the staging rules is
the quickest way to g uarantee that the software test results are not repeatable.
The other way to look at the staging alternatives is to conclude that “quick fi xes”
via manual staging are neither quick nor helpful. If manual staging is problematic
for testing, then it is a disaster for production. Too many companies still allow their
software developers to place untested fi xes directly into production and are amazed
when the application crashes. [42] The irony of the decision is that it takes more time
and resources to correct and test the quick fi x than it would have taken to correct the
original problem the right way in the fi rst place.

10.8 PUTTING THE TESTING ENVIRONMENT
IN PERSPECTIVE

All too often, the testing environment is an afterthought of the development project
that is already tight on budget and computing resources. The fi rst reaction is to give
the test team “orphan” equipment that nobody else wants or uses. The consequence
is test failures on the tester computers that cannot be recreated on the developer
computers because of the disparity in computing resources. It does not take many
of these false test failures for all testing results to be completely dismissed by the
project. Of course the true test failures also become dismissed.
The “pay me now or pay me later” axiom certainly applies to this situation. The
development project really has two choices:

1. Plan to provide an appropriate testing environment separate from all other

project environments to get the best possible test results throughout the
project
Plan to scrimp on the testing environment and likely experience end user dis-
covered defects that cost much more to diagnose and fi x than an appropriate
testing environment would have in the fi rst place.

2.

10.9 SUMMARY

A testing environment allows the testers to observe the execution results that the
customer or user will actually experience in production before the software is