Hardware Reference
In-Depth Information
What Is a Test Case? Revisiting the Software
Test Case Concept
Dani Almog and Tsipi Heart
The Department of Industrial Engineering and Management
Ben Gurion University of the Negev
almog.dani@gmail.com, heart@bgu.ac.il
Abstract.
Since the 1980s the term "Test Case" (TC) has been recognized as a
building block for describing testing items, widely used as a work unit, metric
and documentation entity. In light of the centrality of the TC concept in testing
processes, the questions this paper attempts to answer are: What are the uses of
TC in software testing? Is there a general, commonly agreed-upon definition of
a TC? If not, what are the implications of this situation?
This article reviews and explores the history, use and definitions of TCs, show-
ing that while extensively used in research and practice, there is no one formal
agreed upon definition of a TC. In this paper we point at undesirable implications
of this situation, suggest four criteria for a 'good' TC definition, and discuss the
benefits accrued from such a definition. We conclude by urging the academic and
professional community to formalize a TC definition for the benefits of the indus-
try and its customers, and strongly believe that this review paves the way to ar-
ticulating a formal TC definition. Such a definition, when widely accepted, will
clarify some of the ambiguity currently associated with TC interpretation, hence
with software testing assessment which relies on TCs as metrics. Furthermore, a
formal definition can advance automation of TC generation and management.
1 Introduction
A research initiated by the US Department of Commerce [1] estimated an annual
economic damage equivalent to $20 - $52 billion as a result of inadequate software
testing infrastructure and processes. The authors classified two primary categories of
damages: damages users incurred because of software malfunction, and damages
associated with software modification, fixing and re-testing. Although published some
six years ago, there is a sound indication that the situation has not significantly im-
proved. Hence, the alarming magnitude of damages caused by inappropriate software
testing merits closer investigation into plausible reasons and explanations to this un-
desirable situation in a quest for solutions and improvement.
Because software testing is a broad topic which cannot be grasped in a single work,
this study focuses on one specific aspect of the software testing domain - the test case
(TC), since TC is a cornerstone in software testing processes, and because, as shown
later on, it is posited that inconsistencies in TC definitions and use throughout the
testing process is perhaps a cause for fundamental flaws.
The questions this paper attempts to answer are: What is the role of TC in software
testing? Is there a general agreement about the definition of TC? If not, what are the
consequences of this situation?
