Peer review has some problems – but the science community is working on it

Peer review is the central foundation of science. It’s a process where scientific results are vetted by academic peers, with publication in a reputable journal qualifying the merits of the work and informing readers of the latest scientific discoveries.

But peer review sometimes gets a bad rap – criticised for a purported lack of transparency, low accountability and even poor scientific rigour.

There’s now considerable movement towards tweaking or even remodelling the peer review system. Key areas of focus include making journal editors more directive in the process, rewarding reviewers, and improving accountability of editors, reviewers and authors.

Read more: Science isn't broken, but we can do better: here's how

Peer review relies on volunteers

The peers in the peer review system are volunteer academics with expertise relevant to the paper being considered. But it’s hard to find suitable volunteers.

Reviewing is more complex and onerous than just rejecting or accepting a manuscript. More often than not, a reviewer suggests additional experiments that authors have overlooked, or challenges the interpretation of some of the data. This initiates a dialogue between author and reviewer aimed at improving the integrity and scientific merit of the paper.

It takes time – at least seven to eight hours per paper done properly, with no remuneration or recognition for the reviewer and hence rarely regarded as a priority in a busy academic schedule. As a result, scientific rigour can be lost when reviews become fast-tracked.

Read more: Not just available, but also useful: we must keep pushing to improve open access to research

On the other extreme, sometimes novice reviewers (perhaps trying to impress the editor) can turn small discrepancies into significant flaws. This presents a breakdown in the fairness of the review process.

Overall, these issues create a limited number of peer reviewers in practise, an outcome that can lead to cronyism.

Delving further into this remaining inadequate pool of reviewers, a significant gender gap is also apparent. Nature reports that less than 20% of its reviewers in 2017 were women.

More accountability from editors

We should demand more of our journal editors.

Editors can become more proactive by rejecting articles that are not at publication standard upon submission, rather than placing the arduous task on a reviewer to be both scientist and copy checker.

To retain and train novice reviewers, clearer evaluation criteria from editors would vastly improve the reliability and quality of submitted papers.

Editors could also engage better in a dynamic dialogue between author and reviewer – digital communication technologies enable real-time global discussions to facilitate streamlined review processes for all involved.

Read more: Bored reading science? Let's change how scientists write

Recognition of reviewers

Traditionally, editors are held up to be a revered part of the peer review process, and reviewers are simply not acknowledged for their contributions. But this is changing.

To promote increased transparency, greater accountability and fairness, open peer review processes list reviewers and editors in addition to authors in each publication.

This is happening now in newly established online journals such as eLife. Independent platform Publons rewards reviewers by listing all peer reviewing and editorial activity to provide evidence of a reviewer’s expert contributions in their field. Publons also runs a reviewer awards program.

Read more: Demasi cleared, but images in science continue to attract intense scrutiny

Similarly, Elsevier has started a “reviewer recognition programme”, extending various rewards and publishing a yearly list acknowledging the contributions of all the reviewers.

This process has been met with criticism by some who insist anonymity guarantees unbiased opinions.

Post- vs pre-publication peer review

It’s now becoming clear that scientific dialogue does not need to stop at the endpoint of publication, and that not all problems within a manuscript may be identified at the time of peer review.

Post-publication peer review in its most validated form, involves a journal such as Frontiers asking academics to perform a published interactive dialogue with authors during the review process, giving a level of accountability and responsibility.

Other journals such as Faculty of 1000 Research, Copernicus and PLOS ONE publish papers with minimal evaluation. This shifts the focus towards post-publication peer review – authors, reviewers and readers critique and comment on the paper to judge its scientific merit in the public domain.

Alternative post-publication peer review platforms such as ScienceOpen invite all scientists registered with digital identifier ORCID to write a review or comment on DOI-linked papers. This facilitates engagement of a large cross-section of the scientific community for dynamic appraisal of a publication’s scientific merit.

Forums such as Pubpeer invite anonymous commentary from anyone in the scientific or general community. This occurs without moderation, openly facilitating the possibility of trolling and abusive behaviour at times culminating in legal action.

Time to try something new?

Peer review is not ready to be retired – but it is primed to change.

A recent trial by eLife intends to radically transform the roles of editor, reviewer and author. According to this model, if a senior editor deems a publication worthy of going to review, this paper immediately qualifies for publication.

Read more: Weakened code risks Australia’s reputation for research integrity

Once under review, an open dialogue between author and reviewers takes place. Upon receipt of reviewers’ recommendations, the authors can decide to continue experiments if advised, retract the paper or publish it. This leaves the author’s decision to the scrutiny of the general scientific community.

This innovation may greatly improve the transparency of open peer review, increase accountability on behalf of all participants and reduce burden on the peer review system. It addresses the three major strategies required for improvement of the peer review system. But is it a step too far, too soon? Time will tell.

The overall goal of debates around peer review and appearance of new publication platforms and approaches is to create a united front of authors, reviewers and editors to uphold scientific integrity.

This is vital not just within academic circles, but also to maintain the reputation of science in the broader community.