Performance Testing - Software Testing: Testing Across the Entire Software Development Life Cycle - page 134

9.3 WORKLOAD EXECUTION TECHNIQUES
Once you have identifi ed the groups and volumes of transactions for each peak
workload test, you need to develop the steps to create that peak in a test environment
so that performance measurements can be taken. There are three traditional steps for
executing performance test workload.
9.3.1 Workload Ramp-up to Peak
Workload ramp-up is the process of initiating enough user sessions to drive the peak
workload. If the peak is 2,000 users, then 2,000 users must be logged on to the sys-
tem in the test environment during the fi rst part of the peak workload window. In
real life, 2,000 users would not simultaneously log on within the fi rst seconds of the
lunch hour. The log ons would be staggered over a period of 5-10 min (more work-
load analysis information is needed here) as users put away their daily business and
prepare to browse during their lunchtimes.
Successful ramp-up without any user transaction activity is the testing objec-
tive of the fi rst step. Said another way, if the peak workload of users cannot even
log on, then workload transaction response time is moot. The log on or launch or
startup user activity may appear simple on the screen; however, this is normally the
time when the largest number of processing resources (memory, fi les, and process
threads) are allocated to the user. Complete resource depletion of the application's
test operating environment occurs frequently during initial attempts to ramp up
active users to peak.
The fi rst time you attempt to ramp up any large number (hundreds) of users,
the application is expected to fail before half of the intended users have been ini-
tiated. Try the ramp-up in small increments of 5 or 10 users to better isolate the
ramp-up failure point. At this time in the development and testing cycle, only a
small number (less than fi ve) of users have ever been launched at any time by either
the developers or testers. The ramp-up will reveal all kinds of resource problems
such as not enough memory, not enough processing threads, or not enough fi le
space.
Once the application has been able to successfully ramp up the peak workload
log on without any user activity, the application is tested for successful ramp- up
within a peak time window, say 10 min in our example. Again, expect the applica-
tion to fail several ramp-up attempts within the required time interval. The resource
starvation issues will shift from memory, fi les, and process units to raw CPU speed
and communication bandwidth that really have not been taxed fully by slow ramp-
up testing at this point. Expect some software under development to never achieve
the required peak workload ramp-up in the required time interval due to inadequate
performance design. If the peak workload within the time interval is a contractual
requirement, then the developers are faced with either a major redesign effort and as-
sociated contract penalties or abandonment of the software and loss of the software
development contract.