Information Technology Reference
In-Depth Information
would be one-time use of the environments. In such project environments,
the investment is made in more general-purpose tools rather than in very
specific environments.
The cost elements of the test environment include infrastructure ele-
ments — servers, licenses, etc., the creation and maintenance of test plans
and test cases, and the creation and maintenance of test data.
The best environment to test the application is the one in which the
software is going to be used by the customer. This may not always be
possible, as the production environments are not fixed and final. Distrib-
uted applications, with multiple installs of the software at different loca-
tions, do not have identical configurations. Different versions of the same
third-party hardware and software solutions may be deployed. It is prac-
tically impossible to recreate all possible hardware, software, data, and
usage characteristics of the production environments in which the software
will be used. The subsets that one selects for the in-house testing envi-
ronments will be based on practical factors such as time, budget, and
logistical feasibility. Sometimes, external testing and certification labs can
be used for the other combinations, or else simulated environments can
be created using ghosting techniques.
Because creating and maintaining test data are expensive, many depart-
ments try to use production data (scrubbed or not) as test data. This has
its pros and cons. A successful run of the software using production data
indicates that the software is likely to work in production. However, most
production data, depending on where it is picked up, has had filters and
gatekeepers upstream ensuring that “bad” data has not entered (e.g., one
may have no records with an empty Account Number). However, bad
data is necessary to test how one's software handles it.
Production data may have its own patterns across time. A sample taken
“off season” is very likely different from one taken during “peak season.”
This could become important when it comes to issues of scalability and
software performance. Also, this is one reason why good test data is
created by hand, primarily using production data as a starting point.
Quality Improvements over Time
Applications that have been around for some time, and have been used
frequently by many customers, are, possibly, cleaner because the defects
observed in such applications have been pointed out to technical support
and have already been fixed. It is not enough that the software has merely
been around in the market. Unless many
active
customers have used the
product, it is unlikely to have improved. One must recognize that product
companies cannot, and do not, replicate all production environments.
