Replication of Software Engineering Experiments - Empirical Software Engineering and Verification - page 87

A.17 Schmidt [35]

Direct Replication. This involves repeating the procedure of a previous experi-
ment. In this replication, the context variables, the dependent variable or subject
selection are open to modification.

Conceptual Replication. This is the use of different methods to retest the hy-
pothesis or result of a previous experiment.

A.18 Tsang and Kwan [66]

Checking of Analysis. In this type of replication, the researcher employs exactly
the same procedures used in a past study to analyze the latter's data set. Its
purpose is to check whether investigators of the original study have committed
any errors in the process of analyzing the data.

Reanalysis of Data. Unlike the checking of analysis, in this type of replication,
the researcher uses different procedures to reanalyze the data of a previous study.
The aim is to assess whether and how the results are affected by problems of
definition, as well as by the particular techniques of analysis. Quite often the
replication involves using more powerful statistical thecniques that were not
available when the original study was conducted.

Exact Replication. This is the case where a previous study is repeated on the
same population by using basically the same procedures. The objective is to keep
the contingent conditions as similar as possible to those of the previous study.
The researcher usually uses a different sample of the subjects. The main purpose
is to assess whether the findings of a past study are reproducible.

Conceptual Extension. A conceptual extension involves employing procedures
different from those of the original study and drawing a sample from the same
population. The differences may lie in the way of measuring constructs, struc-
turing the relationships among constructs, analyzing data, and so forth. In spite
of these differences, the replication is based on the same theory as the original
study. The findings may lead to a revision of the theory.

Empirical Generalization. In this replication, a previous study is repeated on
different populations. The researcher runs an empirical generalization to test
the extent to which the study results can be generalized to other populations. It
follows the original experimental procedures as closely as possible.

Generalization and Extension. The researcher employs different research proce-
dures and draws a sample from a different population of subjects. The more impre-
cise the replication, the greater the benefit to the external validity of the original
finding, if its results support the finding. However, if the result fail to support the
original finding, it is dicult to tell whether that lack of support stems from the
instability of the finding or from the imprecision of the replication.