Replication of Software Engineering Experiments - Empirical Software Engineering and Verification - page 85

Differentiated Replication. It involves deliberate, or at least known, variations in
fairly major aspects of the conditions of the study. The aim is to extend the range
of conditions under which the result still holds. Exploring a result with deliber-
ate variations in the conditions of observation is the essence of generalization.
According to the authors, there are three reasons for running a differentiated
replication:

1. Use different methods (different measuring instruments, analysis procedures,
experimental setups, and/or investigators) to reach the same result (trian-
gulation),
2. Extended the scope of the results,
3. Define the conditions under which the generalization no longer holds.

A.13 Lykken [69]

Literal Replication. This involves exact duplication of the first investigator's sam-
pling procedure, experimental conditions, measuring techniques, and methods of
analysis.

Operational Replication. One strives to duplicate exactly just the sampling and
experimental procedures given in the first author's report. The purpose of op-
erational replication is to test whether the investigator's “experimental recipe”
the conditions and procedures he considered salient enough to be listed in the
“Methods” section of his report will in other hands produce the results that he
obtained.

Constructive Replication. One deliberately avoids imitation of the first author's
methods. To obtain an ideal constructive replication, one would provide a com-
petent investigator with nothing more than a clear statement of the empirical
“fact” which the first author would claim to have established.

A.14 Mittelstaedt and Zorn [67]

Type I. The replicating researcher uses the same data sources, models, proxy
variables and statistical methods as the original researcher.

Type II. The replicating researcher uses the same data sources, but employs
different models, proxy variables and/or statistical methods.

Type III. The replicating researcher uses the same models, proxy variables and
statistical methods, but applies them to different data than those used by the
original researcher.

Type IV. In this replication, different models, proxy variables and statistical
methods are applied to different data.