Reimagining Research

We conduct research and development of innovative possibilities for how the research culture could operate.

Reimagining research includes innovation and initial evaluation of new approaches to conducting, consuming, and evaluating research. These test new policies, practices, and solutions that could inform the development of scalable solutions supporting lifecycle open science.

Featured Projects

Lifecycle Journal

Lifecycle Journal is a research and development pilot that reimagines scholarly publishing by enabling researchers to share work at all stages of the research lifecycle and receive community-driven evaluations—shifting scholarly credit toward transparency, rigor, and continual refinement rather than a single static publication outcome. It also provides a platform for testing and comparing independent, innovative research assessment methods.

Predicting Replicability Challenge

The Predicting Replicability Challenge is a research initiative that advances automated methods for evaluating the trustworthiness of research claims by combining machine assessments with human expert review to make scalable, transparent credibility evaluations of quantitative empirical studies. The initiative helps test whether confidence in research can be assessed more efficiently and at greater scale.

Ongoing Activities

These ongoing activities reflect current areas of work, including pilots, partnerships, and applied research designed to explore possibilities for how research can be conducted, assessed, and rewarded.

Registered Revisions

Registered Revisions is a collaborative project testing a peer review innovation in which authors prepare predefined revision plans when reviewers request additional analyses, and journals grant in-principle acceptance to those plans to promote transparency and reduce questionable research practices during the revision process. It introduces greater transparency and structure into revisions involving new data or analyses, helping limit questionable research practices during peer review.

Global Flourishing Study

The Global Flourishing Study is a large international longitudinal research collaboration producing data on human well-being from over 200,000 individuals in more than 20 countries, enabling global analysis of what factors contribute to flourishing while promoting open research practices like preregistration. Preregistration serves as a pathway for early data access, supporting credible, transparent, and reusable research with a major shared dataset.

Journal-Funder Partnerships

COS organizes partnerships between funders and journals to introduce and promote practices, like Registered Reports, in research domains to increase capacity and engagement with open science. For example, the consciousness initiative provides grants for research on consciousness to investigators who receive in principle acceptance for a Registered Report at an eligible journal. These partnerships test how funding and publishing incentives can be aligned earlier in the research process to support rigor and transparency.

Mediated Access to Industry Data

There are several challenges to sharing and using important industry data. We are conducting a pilot with Meta to mediate availability of Instagram data for academic researchers investigating adolescent well-being using the Registered Report publishing model. The approach is intended to address conflicts-of-interest and rigor by conducting academic peer review of proposed research in advance and making public commitments to the design and analysis plan prior to collecting and gaining protected access to the research data. This pilot tests a more rigorous and accountable model for making valuable but sensitive data available for independent research.

How We Approach This Work

We prioritize two domains in reimagining research.

Research assessment and evaluation
We are testing new ways to promote and practice transparency of both the research and its assessment, with an emphasis on opening up the research lifecycle so that all components of research are evaluated and rewarded. Also, we are pursuing and supporting innovation and diversification of the metrics and indicators of research trustworthiness through machine, human, and empirical methods

Partnership models for research rigor and transparency
We test models that help journals, funders, institutions, and data holders support more rigorous and trustworthy research. This includes partnerships, culture-change strategies, and mediated access approaches for sensitive or proprietary data.

 

Archived Activities

These archived activities reflect a longer portfolio of metascience research examining credibility, transparency, and innovative methods across different fields and methods. While these projects are no longer active, they continue to inform COS’s current research, products, and solutions.

Registered Reports Quality Evaluation

Observational evaluation of rigor and quality of Registered Reports compared with standard reports by same authors or on similar topics published in the same journals. Published in Nature Human Behaviour (Soderberg et al., 2021) and posted in MetaArXiv (https://doi.org/10.31222/osf.io/7x9vy).

Registered Reports Impact Evaluation

Observational evaluation of impact and citations of Registered Reports compared with standard reports by same authors or on similar topics published in the same journals. Posted in OSF Preprints (Hummer et al., 2019).