Registered Reports is a publishing format that emphasizes the importance of the research question and the quality of methodology by conducting peer review prior to data collection. This format is designed to reward best practices in adhering to the hypothetico-deductive model of the scientific method. It eliminates a variety of questionable research practices, including low statistical power, selective reporting of results, and publication bias, while allowing complete flexibility to report serendipitous findings.
COS hosted a webinar on June 16, 2022 to review the purpose, structure, and benefits of Registered Report and explore current funding initiatives for researchers interested in conducting research using the Registered Reports model including the Funding Consciousness Research with Registered Reports initiative and The Open Science of Religion project.
Speakers included:
- Michiel van Elk, Associate Professor at Leiden University, Associate Editor of The International Journal for the Psychology of Religion
- Lara H. Engelbert, Ph.D. Candidate, Department of Experimental and Applied Psychology, VU Amsterdam
- Olivia Lowrey, Senior Project Coordinator at the Center for Open Science
- David Mellor, Director of Policy at the Center for Open Science
Mellor kicked things off by providing an overview of the Registered Reports publishing format.
“[Registered Reports] pushes peer review into two stages,” Mellor said. “The first stage of peer review takes place right as the study design process is occurring.”
Mellor walked through the Registered Review process, starting at the first stage of peer review where authors submit the introduction, proposed methods and analyses, and pilot data (if applicable). At this stage, reviewers evaluate if the hypothesis is well founded, if the methods and proposed analyses are feasible and sufficient, and if there are quality control checks. Once evaluated, the proposed study is granted an “in principle acceptance” – a promise to publish regardless of outcome. Stage two of peer review occurs after the study is conducted and includes author submission of introduction and methods, results, and interpretation. The evaluation at that point asks if the positive control succeeded and if the conclusions are justified by the data. Questions regarding the novelty of results are excluded in this stage of the peer review evaluation.
There are over 300 journals that offer the Registered Report format across various disciplines.
Engelbert discussed her personal experience with publishing in the format, highlighting the importance of choosing rigorous analysis techniques and the relevance of choosing appropriate methods and reporting results in a transparent way. Engelbert pointed to publication bias, the replication crisis, and methodological pitfalls existing across scientific disciplines, and how this has shaped her thinking.
“Open science, especially the Registered Reports format, is one of the most effective tools to solve these problems in my view,” Engelbert said. “To me, Registered Reports have really been the scientific Holy Grail – a way of conducting research that prioritizes transparency, reproducibility, thoughtfulness, and eliminates publications bias towards only publishing positive results.”
Lowrey started the conversation around current funding mechanisms, including Funding Consciousness Research with Registered Reports initiative. With generous funding from the Templeton World Charity Foundation and in collaboration with the Association for the Scientific Study of Consciousness (ASSC), COS is leading an initiative awarding a total of $1.125M to consciousness researchers who use the Registered Reports process to publish their papers in participating journals.
“We will be administering 30-75 awards through this initiative in the range of $15,000 to $20,000 USD with some awards in the $50,000 range if exceptional proposals are submitted that require greater funding,” Lowrey said.
To be eligible for this grant, the applicant must use the award to fund an empirical study in consciousness research, reside in an eligible country, and engage in open science practices. Grant applications are accepted on a rolling basis and will be closed when funds are exhausted or by the summer of 2023.
The Open Science of Religion project is another current funding mechanism, as highlighted by van Elk.
“The Open Science of Religion project offers funding for conducting Registered Reports studies in the field of psychology of religion and spirituality,” van Elk said.
The call for proposals for the project, funded by the John Templeton Foundation, runs from 2022-2024. Awards of $20,000 USD are available for 18 studies. Requirements include a cover letter, a manuscript including introduction, methods, and analysis plan, a timeline, budget estimates, and a statement of institution support.
