Information Technology Reference
In-Depth Information
7.5 Performance
7.5.1 Performance as an Acceptance Criterion
When performance becomes an object of acceptance this may cause some
difficulties. On the one hand there is really no special test script in the usual
sense for testing. On the other hand performance itself can be extremely critical
for sequences of process steps viewed by the business end user with respect to the
standard he sets. When the execution of certain functions takes an unacceptably
long time the business itself will be seriously impaired if not made impossible
altogether. For these reasons performance can become an acceptance criterion of its
own. In this context it should be regarded just as any other ordinary test case for the
final acceptance evaluation of the software to be implemented.
7.5.2 Benchmarks
The real problem, however, is not the acceptance criterion itself but the yardstick to
be employed. The subjective notion of a user sitting in front of a silent screen and
waiting for the return of the cursor—the pure response time aspect—is not suffi-
cient in most cases to represent a measurable quantity. For this reason normally no
online functions or GUIs will serve for test cases but rather the duration of batch
jobs, which can be quantified. The determination of CPU time or elapsed time is not
difficult since these are recorded by the system itself. The difficulty, however, lies
in their interpretation.
Ideally the test system represents not only the complete production configura-
tion but also simulates all concurrent processes normally running on the
production machine—including the corresponding user load during day and
night time. But such a constellation is often hard to achieve with regard to
costs. Furthermore doubts may continue once load distribution becomes a subject
of discussion.
For these reasons one usually contends oneself with building a configuration in
the following way:
Representative data volume as a basis for extrapolation,
Power dimensioning of the system regarding central processor and memory
permitting scalability.
The measured times will be extrapolated via an agreed upon algorithm after the
test runs.
