Reading and Reviewing - Writing for Computer Science - page 26

Contribution
Contribution is the main criterion for judging a paper. In broad terms, a paper is a
contribution if it has two properties: originality and validity .
The originality of a paper is the degree to which the ideas presented are signifi-
cant, new, and interesting. Most papers are to some degree extensions or variations
of previously published work; really groundbreaking ideas are rare. Nonetheless,
interesting or important ideas are more valuable than trivial increments to existing
work. Deciding whether there is sufficient originality to warrant publication is the
main task of the referee. Only a truly excellent presentation, thorough and written
well, can save a paper with marginal new ideas, while a revolutionary paper must be
appalling in some respect to be rejected.
When evaluating the significance of a contribution, it is helpful to consider its
effect, or impact : that is, to judge how much change would follow from the paper
being published and widely read. If the only likely effect is passing interest from a
few specialists in the area, the paper is minor. If, on the other hand, the likely effect
is a widespread change of practice or a flow of interesting new results from other
researchers, the paper is indeed groundbreaking.
That some ideas appear obvious does not detract from their originality. Many
excellent ideas are obvious in retrospect. Moreover, the ideas in a well-presented
paper often seem less sophisticated than those in a poorly presented paper, simply
because authors of the former have a better knack for explanation. Obviousness is
not grounds for rejecting a paper. The real achievement may have been to ask the
right question in the first place or to ask it in the right way, that is, to notice that
the problem even existed. Organization of existing ideas in a new way or within an
alternative framework can also be an original contribution, as can reevaluation of
existing ideas or methods.
The validity of a paper is the degree to which the ideas have been shown to be
sound. A paper that does no more than claim from intuition that the proposal should
hold is not valid. Good science requires a demonstration of correctness, in a form that
allows verification by other scientists. As discussed in Chap. 4 such a demonstration
is usually by proof or analysis, modelling, simulation, or experiment, or preferably
several of these methods together, and is likely to involve some kind of comparison
to existing ideas.
In the area of algorithms, proof and analysis are the accepted means of show-
ing that a proposal is worthwhile. The use of theory and mathematical analysis is
one of the cornerstones of computer science: computer technology is ephemeral but
theoretical results are timeless. Their very durability, however, creates a need for
certainty: an untrustworthy analysis is not valuable. Thus a paper reporting exper-
imental work can be a significant contribution. The experiment, to be of sufficient
interest, should test behaviour that had not previously been examined empirically, or
contradict “known” results.
Demonstrations of validity, whether by theory or experiment, should be rigorous:
carefully described, thorough, and verifiable. Experiments for assessment of algo-
rithms should be based on a good implementation; experiments based on statistical