How to Review a Scientific Paper
Purpose of this document
This document is intentionally brief. It is intended as a combination checklist and reminder. For more complete overviews of the review process, see the delightful document linked to in the acknowledgments, and the references therein.
This guide is not meant to be a straight-jacket, but rather something to turn to when you don’t know how to start writing your review, or to check when you’re not sure whether you’ve finished it.
What is in a review?
A scientific review consists of two parts: a confidential cover letter and the anonymous referee’s report.
- The cover letter is addressed to the journal editor, and contains information that will not be forwarded to the authors. This includes:
- your name.
- the paper’s title, authors, and code number.
- a recommendation (accept, reject, etc.) and brief justification.
- No sneak attacks!
The cover letter is also the place for you to
- describe your expertise in the subject area, especially if the paper is not in precisely your own line of research.
- say how confident you are of your views of the paper.
- mention how much time you put into this review.
- if you did not actually check equations in the paper, this is the place to say so.
- remind the editor of any potential conflicts of interest you may have (naturally you mentioned these earlier, before agreeing to review the paper!)
- acknowledge anyone who helped you with the review.
- include personal correspondence with the editor.
The recommendation belongs in the cover letter, rather than the review proper, because then your review can be forwarded to the author directly, without editing, even if your recommendation is not followed.
- The referee’s report is usually forwarded to the authors, and sometimes to other reviewers. It can typically be divided into a number of sections:
- summarizes the paper succinctly and dispassionately. This is not the place to criticize, but rather to show that you understood the paper, and perhaps discuss how it fits into the big picture.
- general comments
- gives the big critical picture, before sinking into the details. This is the place to take a breath, keep your perspective, and explain what the papers weaknesses are and whether they are serious, or intrinsic to our current state of knowledge, or whatever.
- constructive criticism
- not only of technical issues, but also organization and clarity.
- table of typos
- and grammatical errors, and minor textual problems. It’s not the reviewer’s job to copy edit the paper, so don’t go out of your way to look for typos. And if the paper is a complete mess, just say so—but please be charitable, especially if English is not the author’s native language.
This document was inspired and influenced by A guide for new referees in theoretical computer science by Ian Parberry. To quote: “Desirable traits in a referee include objectivity, fairness, speed, professionalism, confidentiality, honesty, and courtesy” and “Before submitting a finished report, a wise referee asks “Would I be embarrassed if this were to appear in print with my name on it?’