Guest Post — Registered Reports Community Feedback: Monitoring Peer Review

Registered Reports are an innovative alternative to traditional scientific publishing. In this format, study proposals are peer reviewed before research has been conducted, and journals commit to publishing the final research based solely on the methodological rigor of the study proposal. By doing so, it seeks to address weaknesses with standard preregistration, guard against various forms of reporting bias and publication bias, and improve research quality. While early evidence suggests that Registered Reports aid reproducibility and research quality uptake of Registered Reports by authors and journals remains limited. This appears to be for myriad reasons, not all of which are well understood.

As someone who advocates for science reform, I find it hard not to take my work home with me. So when a friend is about to start a new piece of research, I can’t help but ask: “have you considered doing it as a Registered Report?”. Unfortunately they often tell me it’s not possible this time. I hear a number of reasons for this: they’ve never written one before and don’t feel confident trying to; they’re part of a bigger research team, where someone more senior decides on the research process; and, lastly, they’re extremely busy - they are deterred by the upfront time costs that the Registered Reports format requires.

To gather more information about how Registered Reports peer review is implemented, and gain insight into what might be stopping the research community from adopting Registered Reports more widely, our team has developed Registered Reports Community Feedback. The website collects anonymous feedback from authors and reviewers of Registered Reports manuscripts, asking a range of questions about their experiences of the peer review process. The website also presents dashboards, which show aggregate quality and speed ratings of Registered Reports across a range of journals. We were inspired by a number of community projects that do something similar for traditional peer review (e.g. SciRev, JournalReviewer, QOAM) as well as well-known platforms in other industries, like Amazon and Yelp!’s ratings systems. Our dashboards, which show aggregate quality and speed ratings of Registered Reports across a range of journals, were inspired by platforms that included rankings such as FDAAA TrialsTracker.

We hope our website will be helpful in three key areas. Firstly, it will help the community in choosing where to submit their Registered Report manuscripts. Secondly, it will incentivise editors and publishers to improve the Registered Reports process at their journals, revealing what they're doing well, what they might do better, and how they can enhance the review process for Registered Reports overall. Finally, the website will have utility as a research tool. It will help our team at Cardiff University answer questions such as:

• How fast can Registered Reports peer review be? What’s the quality like?
• How do Registered Reports compare to traditional peer review?
• What are the key predictors of positive/negative experiences around Registered Reports from an author perspective?

You can support this work by sharing your experiences of Registered Reports. If you've been an author or reviewer of a Registered Report manuscript at Stage 1 and/or Stage 2, you can find our survey here (takes 5-10mins). 

You can also view our dashboards, to help make your next decision about where to submit a Registered Reports manuscript.

If you edit for a journal that offers Registered Reports, you can point your authors and reviewers to our website.