Civil Engineering Reference
In-Depth Information
6.1.5
Verification and Validation of Test Systems
Test systems are implemented following the requirements given by the tests to be
executed as well as by general requirements specified for the environment of the
test object. From these requirements, functions or features of the test system are
derived which must be implemented correspondingly. Most important for trustwor-
thiness of test-system implementations is that all functions of the test systems are
checked on functional correctness, such as communication with the test object, ex-
ecution of TCs, derivation of the test verdict and logging of those data.
This is done in the verification phase of the test system, which should be formal
and carefully planned as well as correspondingly well documented. Any one of the
functions should be tested individually. Especially the decision thresholds for auto-
matic evaluation methods must be checked, because they directly influence the test
result. However, the generation of individual stimuli for either software or hardware
interfaces also must exactly match the expectations. The verification process proves
that all test means, all “test tools” operate reliably and as expected. Therefore veri-
fication is the precondition before applying a (verified) test tool on a component/
module/system which is to be tested.
Not only is the test system itself requested to operate correctly and to accomplish
its intended purpose, but also the TCs, which are executed on the test system, must
be applied appropriately. At this point an important issue follows, which is the check
whether the test system as well as the implemented tests together are well suited for
their operational range. Along with its verification it was proven that the functions
of the test system work as anticipated. Now the focus is on its intrinsic value, plau-
sibility and applicability. These features are checked in a validation process.
As widely known TCs result from appropriate combinations of various test func-
tions for stimuli generation, monitoring and evaluation. It must be proven that this
is the case for any one of the implemented TCs. In particular it must be checked
whether the combined test functions are applied in an appropriate order in conjunc-
tion with the correct parameters and the correct timely context, in order to imple-
ment the given test scenarios of the test specification. The correct stimuli must be
applied at the correct point of time to the correct interfaces of the test object, as well
as all of its relevant interfaces must be monitored meanwhile in the predefined time
window. Although, known from the test scenario and component specification, the
correct reaction of a conforming implementation always is predictable, but as how-
ever the chances for erroneous behaviour practically are indefinitely high, test runs
must be independent from the correct reaction of the test object or, more generally,
test runs must not be dependent on how the test object may react. Otherwise, in case
of a failure this could result in blockades and thus disturb the test process.
A test is validated if it is suited to prove that testing of the requirements of the test
object is accomplished. A sequence of test functions definitely may be inappropriate
to prove a distinct requirement, equivalently to a behaviour. As such, a test is useless
if, e.g. for checking the error-detection feature of a test object, the timely sequence of
test steps reads the error memory of the test object just before the error is stored. Ac-
cording to the component standard, the test object ought to store the error in the error
