Streamlining peer review workflows
Automation requires nuance. At its core, peer review is relational: that idea of evaluating the work of your peers gives both reviewer and reviewee equal status. If we took the people out of the process, then peer review would be something else entirely.
Impact of technology
Where there are general, administrative tasks, there is always a push toward automation. The fewer times manual uploads and downloads have to happen, or manual triggers to kickstart a process, the faster publishing workflows can move. The peer review process is notoriously complex and hard to navigate.
For our part, we’re always thinking about how we can relieve the pressure on editorial teams and make the process simpler without sacrificing the necessary quality assurances peer review provides. Recently, we’ve launched an Auto-Assign Reviewer feature, which allows managing editors to save time assigning individual pieces to individual reviewers. This type of step in the process is perfect for automation, because when given a set of parameters, technology can make these assignments instantaneously.
Impact on people
Automation is supposed to make things easier for the people involved. Sometimes it can feel threatening, as though the machines will take over human roles. But, as we said at the beginning, there’s no peer review without peers.
Automation should make the process smooth and seamless, and remove human error that occurs when we’re rushing or trying to speed past these simple administrative tasks to the more important, and inspiring, parts of our work. Automation is like having an assistant that just happens to be a machine, one who can perform a series of tasks at scale (a scale which the scholarly publishing world needs to prepare for).
Pros and cons of peer review automation
With so many strains on the peer review system, from scale, to integrity, to speed, the question may not be whether to automate, but instead how to automate effectively so we don’t lose the core of what makes peer review impactful.
Pros of peer review automation
-
-
-
A consistent experience: evaluations will always be the same, meaning every author has a similar experience. AI has the potential to reduce bias in the review process, but it would require significant oversight as learned biases still impact AI.
-
-
A faster process: Processing papers, and advancing papers from one stage to the next, can happen much faster with some automated steps. With this speed, we can reduce time to publish and accelerate scientific breakthroughs.
-
A more cost-effective system: Time is money, so any savings on time will translate into financial gains. (When we asked ChatGPT about the pros of automation, it said that automated systems are less expensive because they do not require payment, but since human reviewers are also often uncompensated, this may represent a gap in its learning.)
-
A scalable output: Automated systems can handle high volume inputs. With tools like our Auto Assign Reviewer feature, technology acts as a coordinator, quickly connecting the right papers to the right people without human intervention.
-
Cons of peer review automation
-
Lack of context: Nuance matters, and without human contribution, automated systems may miss key pieces of context and fail to evaluate broader impact or importance.
-
Lack of expertise: Focus on the learning part of machine learning. We need to input information into the technology in order for it to be effective.
-
Lack of feedback loops: The way in which an AI provides feedback to authors might not be the most constructive or relevant, meaning those authors would not benefit from the learnings. In addition, AI requires validation from humans to learn that it is making the right decisions.
-
Lack of ‘peer:’ As we’ve said, it's not peer review without the peer. The human touch of peer review matters. Research integrity relies on the trust inherent in the peer review process. Reviewing requires a great deal of nuance, like when evaluating papers that are written by non-English language speakers or first time authors. The community nature of peer review is a critical part of its strength.
As with so many evolving areas in scholarly publishing, we need to find a balance. There are steps within the peer review workflow that can be automated. As we develop automations in peer review together, we can think of it like a switchboard operator, pushing papers from one reviewer's inbox to the next, or collecting feedback and sharing it with the editor automatically.
No matter what direction peer review workflows go, Morressier is always thinking about them holistically, as a critical part of the infrastructure of publishing. We prioritize streamlining workflows and improving the user experience for authors, reviewers, and editors, while making sure that every new feature we develop removes the potential for error and improves the integrity of the final piece of research.
