Publication pre-commitment devices such as preregistration and Registered Reports (RRs) have been developed as methods to reduce publication biases, prepublication biases (e.g. p-hacking and HARKING), and other questionable research practices. Whether they achieve these ends and whether there are unintended consequences of these interventions remain areas of active research.
The Center for Open Science (COS) will contribute to this growing body of evidence by conducting a randomized trial of Registered Reports in conjunction with the controlled data release of Global Flourishing Study (GFS) to examine the impact of preregistration and registered reports on the research pipeline.
Researchers can normally gain early data access to GFS data if they submit either a preregistration or a Registered Report submission. In this trial, we are asking participants to have their choice of data access pathway (preregistration or RR) be randomly assigned. After randomization, researchers will proceed as normal through their GFS research project, meaning submitting a preregistration or stage 1 Registered Report to the GFS registry before analyzing the data. Additionally, participants - as well as anyone who submits to the GFS registry - have the opportunity to share their experiences and beliefs with preregistration and/or Registered Reports, as well as record key project outcomes through a set of periodic surveys.
This naturalistic trial design allows us to observe the research and publication process for Registered Reports and preregistered research projects and will help inform the scientific community on the strengths, limitations, and unintended consequences of these open science practices.
Preregistration and Registered Reports could be potentially transformative to how research is conducted.
Participating in this trial is an opportunity to learn about these mechanisms in a supported setting and contribute to their evidence base. Watch the webinar below to learn more.
Watch this webinar to learn more about the randomized trial and how you can participate.
The Global Flourishing Study (GFS) is a 5-year longitudinal data collection and research collaboration. This $43.4 million initiative includes data collection for approximately 240,000 participants from 22 geographically and culturally diverse countries, and measures what makes for a flourishing life in 6 areas:
One wave of data will be released each year, and each wave will become publicly accessible a year after its initial controlled release. However, researchers can get immediate access to a wave as soon as it is released by submitting a preregistration or a stage 1 Registered Report to COS. Learn more about GFS.
Researchers can enroll in this trial now by filling out this form. There is no deadline to enroll in the trial, and participants may apply for access to subsequent waves of GFS data.
Sample GFS data will be made available in mid-August 2023, and the first wave of data will be ready for early release in late 2023.
This research is funded by the NSF (grant #2152424), and has been approved by the University of Virginia Institutional Review Board (#5976).
This trial has two components. The first and main component is the randomized trial in which participants are randomly assigned to submit a preregistration or a stage 1 Registered Report of planned analyses to the GFS registry. The second component is a periodic survey that anyone who submits to the GFS registry can enroll in - including those who are not part of the RCT - that records publication pipelines and project outcomes, as well as beliefs and experiences with preregistration/registered reports.
Researchers who:
Each participant may enroll only a single project in this RCT.
If you've already made a submission to the GFS registry, or are uninterested in being randomized but still want to be part of a growing evidence collection on pre-commitment devices, you can still opt into the periodic surveys. All GFS researchers will receive an invitation via email after uploading their submissions to the GFS registry.
You will be eligible for co-authorship on the final manuscript of this study if you agree to be randomized and complete all of the below requirements:
Participants are free to drop out of the trial at any time, including forgoing their assignment and/or ceasing to participate in surveys. Withdrawing from the trial will not affect their access to the GFS data in any way.
Questions about the trial can be directed to Noah Haber (noah@cos.io) and Macie Daley (macie@cos.io)
Fill out this form to enroll in the trial. The form will take about 10 minutes to complete and includes a baseline survey and informed consent. After filling out the survey, you will receive your random assignment via email.
All GFS researchers will be invited to opt into our surveys via email after uploading a submission to the registry.
You can still participate in the periodic surveys even if you have already made a submission to the GFS registry or you are not interested in participating in the trial.
What is the difference between a preregistration and a stage 1 Registered Report?
A preregistration and a stage 1 Registered Report (RR) are closely related. Both mechanisms are a way of recording your hypotheses, methods, and/or analyses of a scientific study before it’s conducted.
A stage 1 RR is submitted for review at a journal, generally before data are collected or analyzed. Once the stage 1 RR undergoes peer review and is given in-principle acceptance, authors can access their data and conduct their analyses.
Conversely, a preregistration does not undergo peer review. When you preregister your research, you're simply specifying your research plan in advance of your study and submitting it to a registry.
Can I access sample data from the GFS dataset and still participate in this trial?
Yes! You may access sample data to inform your research project and can opt into the randomized trial after viewing the sample data. For more information on how to access the sample dataset, please click here.
If I am randomly selected to the Registered Reports arm, will I be forced to go through a Registered Report if things don’t work out?
No! We merely ask that you make a serious attempt to go through a registered report pathway with your project. If you choose to abandon that pathway and instead pursue your project through preregistration or other pathways, that is ok! This trial is primarily about experiences and timelines through the registered report pathway, and your experiences (whatever they are) are the primary result of this trial.
Does my stage 1 registered report need to get in-principle acceptance (IPA) from a journal before I can access the GFS dataset?
No, you do not need to receive IPA in order for COS to grant your access to the dataset. As long as you have submitted your stage 1 RR to a journal and meet all other GFS requirements, you can be approved for access.
However, in most cases, journals restrict researchers from accessing the data for their registered report until after IPA has been granted. Therefore, if you have submitted a stage 1 RR, we strongly recommend that prior to requesting access to or viewing the GFS dataset you reach out to the journal you are working with to discuss their rules about viewing data before IPA has been granted.
Which journals accept registered reports?
See a curated (but not exhaustive) list of journals that accept registered reports here.
Is the Registered Report pathway more work than the preregistration pathway?
Both preregistration and Registered Reports shift the typical research process workflow, though not necessarily by adding more work. Both mechanisms guide researchers to thoroughly plan their projects before conducting their analyses, meaning the bulk of the work must happen at the beginning of a study rather than later on in the research process.
Registered Reports take the shift in the typical workflow a step further by requiring that a project’s hypotheses, methods, and/or analyses go under review before data is analyzed. All papers submitted for publishing must undergo review, and in the RR process, that step occurs earlier than typical practice. This could reduce overall time to publication because once IPA is received, researchers do not need to spend time submitting to multiple journals until their manuscript is accepted. However, we do not have strong evidence at this time as to how Registered Reports impact timelines and this is one of the key insights we are hoping to gain in this study.
What are the requirements for being listed as a co-author on the final manuscript of this trial?
The requirements for co-authorship on this project are:
Participants must complete all four of the above requirements to be eligible for co-authorship.
How often are participants surveyed?
Participants will be surveyed every three months until the end of their projects, or until they indicate they would no longer like to participate.
I have multiple collaborators working on this project. Do we each need to enroll in this trial individually? How does this affect eligibility for co-authorship?
This trial intends to take guidance from ICMJE’s author guidelines.
We ask that one individual per project enrolls and participates in this study. Ideally, this will be the individual that makes the submission to the GFS registry. The individual who enrolls and participates in the trial - given that they meet all of the requirements for co-authorship - will be eligible to be listed as co-author on the final manuscript of this trial. This will be reviewed on a case-by-case basis.
210 Ridge McIntire Road
Suite 500
Charlottesville, VA 22903-5083
Email: contact@cos.io
Unless otherwise noted, this site is licensed under a Creative Commons Attribution 4.0 International (CC BY 4.0) License.
Responsible stewards of your support
COS has consistently earned a Guidestar rating of Platinum for its financial transparency, the highest rating available. You can see our profile on Guidestar. COS and the OSF were also awarded SOC2 accreditation in 2023 after an independent assessment of our security and procedures by the American Institute of CPA’s (AICPA).
We invite all of our sponsors, partners, and members of the community to learn more about how our organization operates, our impact, our financial performance, and our nonprofit status.
