The pitfalls and potential of peer review
In our recent whitepaper, we start off with the quote “peer review is the worst method of publication, except for everything else.” We see such potential in this segment of publishing workflows, and we admire the commitment and care that each peer reviewer gives.
But that doesn’t mean peer review is sacred, or can’t be changed to keep up with the publishing landscape today. Let’s explore some experiments and critical moments in the history of this powerful process.
Peer review in the mid-20th century
While the first recorded instance of pre-publication peer review dates back to 1665, from the Philosophical Transactions of the Royal Society, most early peer review was done by solitary editors or small committees until the end of the 19th century.
External reviewers were not commonplace until the mid-20th century. In fact, Nature only instituted a formal peer review process in 1967!
The “Sokal Hoax”
As various fields in the sciences, social sciences, and humanities have evolved and emerged, the language they use has become more prone to jargon and even buzzwords.
Physicist Alan Sokal put this to the test in 1996. He successfully published a paper in a cultural studies journal, Social Text, which argued that gravity was a social construct. Using the fashionable language of the time, he was able to evade being caught in the peer review system. At the time, the journal’s reputation suffered, but it maintained a focus on the subjectivity of knowledge. That very subjectivity remains a part of the peer review process.
The “Peer-Review Ring”
In 2013, Springer Nature identified a “peer review ring” among their reviewers. These researchers worked together to create fake reviewer identities in order to review their own papers. When the publisher noticed that several reviews were coming from the same IP address, they began an investigation.
As soon as this fraudulent peer review ring was identified, the researchers were banned from publishing in Springer journals, and the papers in question were retracted, restoring the quality and high levels of integrity to their publications.
The “Famous Names” Experiment
Much has been written about the Matthew Effect, on science funding, citation impact, and more. In the peer review process, blind and double-blind peer review are in place to try and avoid that bias in peer review.
Despite these interventions, bias in peer review still persists. Whether it's thanks to the prominence of the researcher submitting their work, or the gender, race, institution, or geography, the scholarly publishing industry constantly works to improve peer review by eliminating bias.
The Peer Review Successes
We would be remiss if we didn’t point out that for every peer review experiment to identify failings or discredit the process, there are countless success stories. It would be impossible to track each and every one of these, but we’ll focus on the research that ultimately led to the 2005 Nobel Prize in Physiology or Medicine.
It is often harder for emerging research to overturn existing research. For Barry Marshall and Robin Warren, that discovery came when they discovered the bacteria that caused stomach ulcers. They faced skepticism and rejection from the scientific community, but persevered until their findings were confirmed through rigorous peer review.
Each of these experiments or attempts to reveal gaps in the peer review workflow are valuable for the scholarly publishing industry. They are all opportunities to improve.
We’re especially energized by the next stage of evolution for peer review - the impact of AI, the increasingly global nature of science, and the power of technology to streamline the process from start to finish.
