Replication of Software Engineering Experiments - Empirical Software Engineering and Verification - page 81

Data Re-analyses. This replication can be defined as an application of previously
published data that is concerned with assessing whether similar findings can be
obtained using a different methodology with the same data or a sub-sample of
the data.
A.4 Finifter [71]
Virtual Replication. The intention is to repeat an original study not identically
but “for all practical purposes” to see whether its results hold up against chance
and artifact. Virtual replications are also frequently conducted to find out how
dependent a result is on the specific research conditions and procedures used in
an original study. To answer this question, one or more of the initial method-
ological conditions is intentionally altered. For example, a survey or experiment
might be repeated except for a change in measuring devices, in the samples used,
or in research personnel. If the initial result reappears despite changes, faith in
the original finding mounts.
Systematic Replication. The emphasis in systematic replication is not on repro-
ducing either the methods or the substance of a previous study. Instead, the
objective is to produce new findings (using whatever methods) which are ex-
pected by logical implication to follow from the original study being replicated.
When such an implication is actually confirmed by systematic replication, con-
fidence is enhanced not only in the initial finding that prompted the replication
but also both in the derived finding and in whatever theoretical superstructure
was used to generate the confirmed inference.
Pseudoreplication. It can be defined according to three main operational vari-
ations: the repetition of a study on certain subsets of an available total body
of real data; the repetition of areal data study on artificial data sets which are
intended to simulate the real data; and the repeated generation of completely
artificial data sets according to an experimental prescription.
A.5 Hendrick [70]
Strict Replication. An exact, or strict, replication is one in which independent
variables (treatments) are duplicated as exactly as possible. That is, the physical
procedures are reinstituted as closely as possible. It is implicitly assumed that
contextual variables are either the same as in the original experiment, or are
irrelevant.
Partial Replication. A partial replication is some change (deletion or addition) in
part of the procedural variables, while other parts are duplicated as in the orig-
inal experiment. Usually some aspect of the procedures is considered “unessen-
tial”, or some small addition is made to expedite data collection.
Conceptual Replication. A conceptual replication is an attempt to convey the
same crucial structure of information in the independent variables to subjects,
but by a radical transformation of the procedural variables. In addition, specific
task variables are often necessarily changed as well.