Information Technology Reference
In-Depth Information
Content of Reviews
Reviewing of papers serves two purposes. The explicit purpose is that it is the mech-
anism used by editors to decide whether papers should be accepted for publication.
The implicit purpose—equally important, and often overlooked—is that it is a means
of sharing expertise between scientists, via comments for the authors. Reviews usu-
ally include other things besides written comments (such as scores on certain criteria,
which are used to determine whether the paper should be accepted), but it is the com-
ments that authors find valuable. The review should make some kind of case about
the paper: whether it is of an adequate standard and what its flaws are. That is, it is
an analysis of the paper, explaining why it is or is not suitable for publication.
There are two main criteria for measuring referees' reviews.
Is the case for or against the paper convincing?
When recommending that a paper be accepted, the editor must be persuaded that
it is of an adequate standard. Brief, superficial comments with no discussion of
the detail of the paper provoke the suspicion that the paper has not been carefully
refereed. A positive review should not just be a summary of the paper; it should
contain a clear statement of what you believe the contribution to be.
When recommending that a paper be rejected, a clear explanation of the faults
should be provided. It is not reasonable, for example, to simply claim without
references and explanation that the work is not original or that it has been done
before—why should the author believe such a claim if no evidence is given?Having
gone to considerable lengths to conduct and present their work, few authors will
be persuaded to discard it by a couple of dismissive comments, and will instead
resubmit elsewhere without making changes.
Is there adequate guidance for the authors?
When recommending that a paper be accepted, referees should describe any
changes required to fix residual faults or to improve the paper in any way—
technically, stylistically, whatever. If the referee doesn't suggest such changes,
they won't get made.
When recommending that a paper be rejected, a referee should consider what the
authors might do next—how they can proceed from the rejection to good research.
There are two cases. One is that the paper has some worthwhile core that, with
further work, will be acceptable. A referee should highlight that core and explain
at least in general terms how the authors should alter and improve their work. The
other case is that nothing of the work is worthwhile, in which event the referee
should explain to the author how to come to the same conclusion. Sometimes the
referee just cannot tell whether there is worthwhile material because of defects in
the presentation. It is helpful to explain to the authors how they might judge the
significance of their work for themselves by, for example, sketching questions the
authors should consider.
There are many reasons why these criteria should be observed. The scientific commu-
nity prides itself on its spirit of collaboration, and it is in that spirit that referees should
