A roadmap for peer review: Revisiting Erica Frank’s 1996 suggestions
This post originally appeared on the SAGE Connection blog on September 13, 2017.
by Michael Blades, Editor of Applied Spectroscopy
In 1996 Dr. Erica Frank wrote a series of editor-reviewer “ideal” communications. Revisiting these suggestions, Michael Blades, Editor of the journal Applied Spectroscopy, explores if over two decades later the notion of the “still-imperfect art” of peer review remains the same today, presenting the guide as a roadmap for the 21st century reviewer.
In 1996, Dr. Erica Frank then at Emory University, now a colleague of mine at The University of British Columbia, published a paper in Preventive Medicine with the title, “Editors’ Requests of Peer Reviewers: A Study and a Proposal”. In the article, Erica described a study into “the way in which reviewers’ opinions are elicited” in the peer review process for scholarly journals. She found that there was substantial non-uniformity in the peer review process both with respect to the expectations of journal editors and the extent of the feedback to the editor and authors provided by peer reviewers. In perhaps the most insightful statement in the article, Erica observed, “peer review is a subtle and still-imperfect art that includes reviews of variable effort and quality”. This is as true now as it was 21 years ago when the article was published.
So although a “still-imperfect art” what is the ideal editor-reviewer communications? In her article Frank made a series of suggestions, based largely on guidelines published by the Council of Biology Editors. To provide a useful roadmap for the reviewer of today, and in the interests of brevity, I have edited these recommendations below, as well as highlighted my top three things to keep in mind from an editor’s point of view:
Proposed ideal for editor–reviewer communications
Reviewers should be asked to:
Maintain manuscript confidentiality
Disclose conflicts of interest
Be objective
Be specific in comments to authors and editors
Respond promptly (and comply with a specified date review completion)
Reviewers should address manuscripts’:
Quality
Importance
Scientific rigor
Novelty
Clarity (in text and in tables/figures/illustrations)
Ethics (including divulged and un-divulged authorial biases)
Specific merits and problems, as contained abstract, key words, introduction, methods interventions/procedures, statistics, hazards, results, interpretations and discussion (including the appropriateness conclusions and statements of study figures, acknowledgments, and citations)
Appropriateness and priority for the journal in question
Appropriateness in its current form (versus for example reduction to a letter to the editor)
Editors may wish to:
Provide thorough instructions for reviewers (especially first time reviewers)
Allow reviewers to share manuscripts with colleagues, in a confidential manner, for assistance with review
Send reviewers copies of others’ reviews
Inform reviewers regarding manuscripts’ final disposition
Ask reviewers to specify other areas in to review
Provide reviewers a token of gratitude
Insights from a journal editor:
Respond immediately to the request. Most peer review systems have a period during which the system waits for a response from the peer reviewer. An immediate response whether positive or negative will speed up the process making it much easier for the editor to manage the peer review.
If you agree to review, please provide the review in a reasonable length of time. The most frustrating thing for an editor and authors is to wait for a peer review that never comes.
If you aren’t able to complete the review, let the editor know rather than making up a review like, “This is a great paper. It should be published”. Typically, this response is code for, “I haven’t read the paper and I am tired of you pestering me – this should make you stop”.
There has been some progress over the past twenty years since Frank wrote the paper. For example many undergraduate and graduate programs require that students be instructed on ethics and ethical behaviour with the hope that the principles learned carry over into their professional life. The blog site Retraction Watch also provides daily reports on retractions of scientific papers and in my opinion is required reading for everyone who is involved in scholarly publishing to create greater transparency around misconduct. In the words of Biologist Jim Woodgett, writing in Nature:
“The scientific community must be diligent in highlighting abuses, develop greater transparency and accessibility for its work, police research more effectively and exemplify laudable behaviour. This includes encouraging more open debate about misconduct and malpractice, exposing our dirty laundry and welcoming external examination. A good example of this, the website Retraction Watch shines light on problems with papers and, by doing so, educates and celebrates research ethics and good practice. Peer pressure is a powerful tool — but only if peers are aware of infractions and bad practice.”
In terms of recognizing contributions to peer review, Publons is a new organization that works with reviewers, publishers, universities and funding agencies to turn peer review into measureable research metrics. Collecting peer review information from reviewers and publishers, Publons produces comprehensive reviewer profiles with publisher-verified peer review contributions that researchers can add to their resume and therefore enabling reviewers to showcase the important work that they do.
Peer review is a flawed process, but until a better, workable methodology comes along most journals and editors will continue to rely on it for manuscript evaluation. Reviewing the manuscripts of others is a lot of work and, for the most part, reviewers take the task very seriously. I would personally like to thank all the peer reviewers who have given their time and effort to support scholarly publishing and all they do in protecting, ensuring and certifying the integrity of the academic record.