Information Technology Reference
In-Depth Information
help others to improve their work. Poor reviews, although saving the referee effort,
make more work for the research community as a whole: if a paper's shortcomings
are not adequately explained, they will still be present if the paper is resubmitted.
Most of all, poor reviewing is self-reinforcing and is bad for scientific standards. It
creates a culture of lacklustre checking of other people's work and ultimately saps
confidence in published research.
In a review recommending acceptance, there is no further chance to correct
mistakes—the referee is the last expert who will carefully examine the paper prior to
its going into print. As noted earlier, only obvious errors such as spelling and punctu-
ation may be caught later, and the referee should check that the paper is substantially
correct: no obvious mathematical errors, no logical errors in proofs, no improbable
experimental results, no problems in the bibliography, no bogus or inflated claims,
and no serious omission of vital information or inclusion of irrelevant text.
In reviews that recommend rejection or substantial revision, such fine-grain check-
ing is not as important, since (presumably) the paper contains gross errors of some
kind. Nonetheless some level of care is essential, if only to prevent a cycle of cor-
rection and resubmission with only a few points addressed each time. Specific, clear
guidance on improving the paper is always welcome. But don't slip into doing the
research for the author. If the work is inept, step back; if the work is strong, your con-
tribution isn't needed; if simple changes will make a real difference, suggest them,
but it is the author's job to take them to completion.
Drafting a Review
First impressions of papers can be misleading. My reviewing process is to read the
paper and make marginal notes, then decide whether the paper should be accepted,
then write the comments to the authors. But often, even in that last stage, my opinion
of a paper changes, sometimes dramatically. Perhaps what seemed a minor problem
is revealed as a major defect, or perhaps the depth of the paper becomes more evident,
so that it has greater significance than had seemed to be the case. The lesson is that
referees should always be prepared to change their minds, and not commit too soon
to a particular point of view.
Another lesson is that positives are as important as negatives: reviews should
be constructive. For example, in the reviewing process it is sometimes possible to
strengthen the paper anonymously on behalf of the author. The reviewing process
can all too easily consist of fault-finding, but it is valuable for authors to learn which
aspects of their papers are good as well as which aspects are bad. The good aspects
will form the basis of any reworking of the material and should thus be highlighted
inareview.
Some referees construct flaws in papers where none exist. For example, an
assessment may include generic statements that could be made almost regardless
of relevance to the paper's topic, such as “the authors have not considered paral-
lel architectures” on a paper about document processing. Other examples are vague
