Hardware Reference
In-Depth Information
software componentization [37, 38] and component-based testing, adding additional
ambiguity to the TC concept.
Some research has focused on
automatic TC generation
, a process requiring TC
formalization [39-42]. As use cases largely reflect functional requirements in the
UML environment, Nebut, Fleurey, Le Traon & Jיzיquel [43] suggested TC
generation from use cases, after incorporating the contract element they claim is a
component essential for translating a use case into a TC. Likewise, test objectives and
sequence diagrams also serve as sources for TC generation. Generally, several works
have developed techniques to generate TCs from UML diagrams, termed Model
Based Testing (MBT), mostly based on transforming use cases and states into TCs
[44, 45]. Although the attempts to automate TC generation resulted in some level of
formalization, the difficulties pertaining to the TC concept were not solved by this
mechanism, since use cases and scripts all suffer from the same fuzziness of defini-
tion regarding size, complexity, number of states, etc.
4.3 TCs as Metrics
During the testing phase, there is a need to manage and control the process, by meas-
uring its size, complexity, and quality, as a minimum. This, however, is easier said
than done, due to reasons brought in the previous sections. Thus, for example when
using the Goal - Questions - Metrics (GQM) method
1
developed by V. Basili and D.
Weis for measurement development, Management strives to find metrics to answer
questions such as 'how long would it take to complete testing?', or 'how much re-
sources should be allocated to testing?', aimed at achieving managerial goals such as
appropriate resource allocation and adhering to schedules. Measures developed to
answer these questions often rely on number of TCs, for example "total number of
planned white/black box test cases run to completion, number of planned integration
tests run to completion, or number of unplanned test cases required during the test
phase" [26, p. 15]. The Software Testing Reliability Early Warning Model for Java
(STREW-J) developed by Nagappan [26] to estimate expected problems as a means
to estimate testing efforts used at least two estimation parameters that are based on
number of TCs: 1)
number of test cases divided by source lines of code (R1)
as an
indication of whether there are too few test cases written to test the body of source
code; and 2)
number of test case divided by number of requirements (R2)
as an indica-
tion of the thoroughness of testing relative to the requirements. Other TC-based met-
rics recommended as reflecting the status of the testing project were number or
percent of TCs run since testing started, number or percent of TCs run since the last
status report, number of percent of TCs that passed since the beginning of the testing
project, number or percent of TCs passed since the last status report, number or per-
cent of failed TCs, total number of open issues or TCs not run [46].
Elsewhere, eight of thirteen reports recommended as tools for testing monitoring
and control were based on TCs count, completion status, results etc. [47]. Further,
these same authors suggested eighteen indicators to monitor the project status, eleven
of which are based on tests or TCs. Two real-world examples of using TCs as the unit
for testing progress monitoring are presented in Figures 4 and 5. Figure 4 illustrates
1
We thank the reviewer for suggesting using GQM as a metric-generation methodology.
