The worst way to judge research?
Since its early days, peer review has had its critics, with Ernest Hart, the editor of the British Medical Journal, famously calling the process, “a laborious and difficult method” in 1893. More recently, Gary Smith notably said, “Once scientific accomplishments came to be gauged by the publication of peer-reviewed research papers, peer review ceased to be a good measure of scientific accomplishments.”
Are the critics right? Currently, about 2.5 million peer-reviewed papers are published annually worldwide. And while there are certainly willing experts ready to review, the supply of reviewers falls short in matching the expanding volume of research.
In a study conducted back in 1982, researchers selected 12 articles that had previously been accepted by prestigious journals, replaced the authors' genuine identities and academic affiliations with fake ones, and resubmitted these identical articles. Only 8 percent of editors or reviewers identified the duplication.
On top of issues of bias and consistency, peer review is slow. After peer review took a collective 15,000 years in 2022, many argue that the pace of the peer review process can hinder innovation and render findings less impactful.
Another pressing concern is the potential for bias in peer review. Reviewers often have access to the names of authors and their affiliations, and many studies have shown that reviews tend to favor or reject articles based on specific demographic factors.
Are we doing enough?
Many believe that the peer reviewer shortage, the rise in retractions, and controversies in peer-reviewed journals all stem from issues such as a lack of incentive for reviewers and a reluctance to innovate the traditional peer review system.
Even still, our industry has made great strides to provide more recognition for reviewers and evaluate research outside the traditional confines of the peer review process.
The different shades of peer review
Reviewer Recognition
Many societies are collaborating with the Web of Science Reviewer Recognition Service, formerly Publons, to provide recognition for peer review. Many researchers, like James Heathers, have also advocated for financial incentives for reviewers. In 2020, Heathers began the 450 movement, stating that $450 is an adequate fee for peer reviewers to charge publishers, even creating a sample contract approved by lawyers for researchers to use when negotiating with publishers. These recognition efforts for peer reviewers not only boost motivation but also contribute to more rigorous evaluations.
Crowdsourcing peer review
Crowdsourcing peer review involves facilitating peer review with a diverse audience that includes researchers, practitioners, and even the general public. This approach seeks to diversify perspectives and speed up the review process. PubPeer has played a huge role in identifying misconduct and fraud cases in research papers through collaborative efforts.
Preprint archives
In the pre-publication model, articles are shared before undergoing formal peer review in the form of preprints, as seen on platforms like arXiv and bioRxiv. Preprints are rapidly rising in popularity, kickstarting instant feedback and dissemination of research findings, and inviting feedback from the wider community.
Post-publication peer review
In the post-publication review approach, articles are published first and then peer reviewed. Journals like F1000Research have embraced this approach, another model that brings speed to scholarly publishing while still checking for quality control.
The Morressier Difference
Since the announcement of our partnership with IOP, both teams have been dedicated to reimagining journal workflows and submission systems. As we set our sights on the future of peer review, how can we transform this process for the better?
Our approach to this problem comes from a service design mindset, combining the objectives of every stakeholder involved and reflecting these viewpoints in tangible software solutions. It champions a human-centric understanding of the entire publishing ecosystem, which is necessary these days given the amount of players within the scholarly ecosystem.
But what might this innovative approach look like when applied to peer review? One possibility is a more transparent, open model that shares the identities of both authors and reviewers, and publishes an article alongside the reviews and comments that improved its quality. Peer review could become more of a dialogue and exchange rather than separate documents of manuscript, edits, and comments.
What about AI-powered systems and automated solutions? They can step in for manual tasks and accelerate the peer review process, leaving the ‘peers’ in the review process to focus on deeper, more substantive feedback.
Another compelling avenue is the concept of decentralized peer review. Blockchain-based systems are taking center stage, recording reviewers' contributions in a transparent way. This approach ensures accountability while minimizing the risk of manipulation, and offers a fresh perspective on how we can boost the integrity of peer review.
Conclusion
Last Peer Review Week, Dana Compton, Managing Director and Publisher, at American Society of Civil Engineers, said,
“Hey, I love peer review as much as the next girl, but there are countless ways it can be improved, from new methods to minimize bias to better technologies and automation. Hopefully that will come with time and money.”
We feel the same way! We believe in the value of peer review and feel confident about its future. But, we must start exchanging the parts that aren’t working for new tools, streamlined workflows, and progressive ways of thinking so we can open the door to exciting new advancements and avoid getting locked in a status quo because it's too hard to change.
Learn more about our vision for peer review here.
