--- title: "Next Steps for the SCORE Project: Spurring Efforts for Assessing Research Credibility" description: Assessing the credibility of research claims is a central and continuous part of the scientific process. However, current assessment strategies often require substantial time and effort. In an effort to accelerate research progress, the Center for Open Science (COS) partnered with the Defense Advanced Research Projects Agency's (DARPA) program Systematizing Confidence in Open Research and Evidence (SCORE) in 2019 to work towards developing and deploying automated tools to provide rapid, scalable, and accurate confidence scores for research claims. image: https://www.cos.io/hubfs/SCORE%20blog%20header.png --- - Our sites: - [OSF](https://osf.io) - [TOP Guidelines](https://www.cos.io/initiatives/top-guidelines) - [* *Bluesky](https://bsky.app/profile/cos.io)[* *Mastodon](https://fosstodon.org/@CenterforOpenScience)[* *Linkedin](https://www.linkedin.com/company/center-for-open-science/)[* *YouTube](https://www.youtube.com/@CenterforOpenScience) - [**  Shop](https://www.redbubble.com/people/OpenScience/shop) - [**  Donate](https://www.cos.io/support-cos) [![Center for Open Science](https://www.cos.io/hs-fs/hubfs/Logos/COS%202024/cos-full-tag-h.png?width=2000&height=293&name=cos-full-tag-h.png "Center for Open Science")](https://www.cos.io) ** ** # Next Steps for the SCORE Project: Spurring Efforts for Assessing Research Credibility May 26th, 2022, [Tim Errington and Andrew Tyner](https://www.cos.io/blog/author/tim-errington-and-andrew-tyner) Posted in: [Replication](https://www.cos.io/blog/tag/replication), [Research](https://www.cos.io/blog/tag/research), [Community](https://www.cos.io/blog/tag/community), [Open Science](https://www.cos.io/blog/tag/open-science), [SCORE](https://www.cos.io/blog/tag/score) ![](https://www.cos.io/hubfs/SCORE%20blog%20header.png) Assessing the credibility of research claims is a central and continuous part of the scientific process. However, current assessment strategies often require substantial time and effort. In an effort to accelerate research progress, the Center for Open Science (COS) partnered with the Defense Advanced Research Projects Agency's (DARPA) program [Systematizing Confidence in Open Research and Evidence](https://www.darpa.mil/program/systematizing-confidence-in-open-research-and-evidence) (SCORE) in 2019 to work towards developing and deploying automated tools to provide rapid, scalable, and accurate confidence scores for research claims. Since then, COS has completed extraction of scientific claims from a stratified sample of social-behavioral science papers. In total 7,066 claims were extracted manually, enabling confidence scores to be assigned by human forecasters and algorithms. Concurrently, COS worked with hundreds of researchers to conduct replications and reproductions on a subset of these extracted claims. The team [leveraged the Open Science Framework (OSF)](https://www.cos.io/blog/large-scale-collaboration-with-osf) for this large-scale collaboration so that materials from the replication and reproduction efforts can be made openly available. The project, now in its final phase, is beginning the transition from working with the research community at large to conduct reproduction, robustness, and replication studies to preparing the project for public release and organizing the data and materials for the wider research community’s use. Over the next six months, COS will be working with close collaborators to compile and enhance the project’s data; continue coding core project outcomes; and draft reports on what we learned about replicability, reproducibility, and robustness, as well as the potential for experts, forecasters, and algorithms to contribute scalable tools for research assessment. Some exciting expansions the SCORE project has made over previous large-scale replication projects (e.g., [Reproducibility Project: Psychology ](https://osf.io/ezcuj/wiki/home/)and [Reproducibility Project: Cancer Biology](https://www.cos.io/rpcb)) is to engage a broad range of [disciplines across the social-behavioral sciences](https://docs.google.com/spreadsheets/d/1accbHCaU4XCAq0iQPgQRdcjgobuMU_L0WjlIKCUf-o8/edit#gid=0). The breadth and depth of SCORE data will be a major contribution to the metascientific field. Finally, the outcomes of this project will lay the groundwork for advancements in algorithmic scoring and will spur future research for assessing research credibility. ### Recent Posts ![Center for Open Science](https://www.cos.io/hubfs/Logos/COS%202024/cos-full-tag-h-w.png) 6218 Georgia Avenue NW, Suite #1, Unit 3189 Washington, DC 20011 Email: [contact@cos.io](mailto:contact@cos.io) - [* *Bluesky](https://bsky.app/profile/cos.io) - [* *Mastodon](https://fosstodon.org/@CenterforOpenScience) - [* *Linkedin](https://www.linkedin.com/company/center-for-open-science/) - [* *YouTube](https://www.youtube.com/@CenterforOpenScience) **Join our mailing list** [![Creative Commons License](https://i.creativecommons.org/l/by/4.0/88x31.png)](http://creativecommons.org/licenses/by/4.0/) Unless otherwise noted, this site is licensed under a [Creative Commons Attribution 4.0 International (CC BY 4.0) License](https://creativecommons.org/licenses/by/4.0/). [Translation Help](https://www.cos.io/how-to-translate) | [Terms of Use](https://github.com/CenterForOpenScience/centerforopenscience.org/blob/master/TERMS_OF_USE.md) | [Privacy Policy](https://github.com/CenterForOpenScience/centerforopenscience.org/blob/master/PRIVACY_POLICY.md) - [**About**](https://www.cos.io/about) - [Team](https://www.cos.io/about/team) - [Board](https://www.cos.io/about/board) - [Partners](https://www.cos.io/about/partnership-framework) - [News](https://www.cos.io/about/news) - [Impact](https://www.cos.io/impact) - [History](https://www.cos.io/timeline) - [Finances](https://www.cos.io/about/finances) - [Compliance and Standards](https://www.cos.io/compliance-and-standards) - [Diversity, Equity & Inclusion](https://www.cos.io/about/diversity-equity-and-inclusion-at-cos) - [Careers](https://www.cos.io/careers) - [**Engage**](https://www.cos.io/engage) - [Tools](https://www.cos.io/tools) - [Training](https://www.cos.io/services/training) - [Community Building](https://www.cos.io/community-building) - [Policy Reform](https://www.cos.io/policy-reform) - **  [Donate](https://www.cos.io/support-cos) - [**Open Science**](https://www.cos.io/open-science) - [Preregistration](https://www.cos.io/initiatives/prereg) - [Registered Reports](https://www.cos.io/initiatives/registered-reports) - [**Research**](https://www.cos.io/services/research) - [SCORE](https://www.cos.io/score) - [RP: Cancer Biology](https://www.cos.io/rpcb) - [RP: Psychology](https://osf.io/ezcuj/wiki/home/) - [Events](https://www.cos.io/events) - [Blog](https://www.cos.io/blog) - [Contact](https://www.cos.io/contact-us) - [Shop](https://www.redbubble.com/people/OpenScience/shop) - [OSF](https://osf.io) - [TOP Guidelines](https://www.cos.io/initiatives/top-guidelines) **Responsible stewards of your support** [![Charity Navigator](https://www.cos.io/hs-fs/hubfs/Logos/charity-navigator.png?width=100&name=charity-navigator.png)](https://www.charitynavigator.org/ein/461496217)[![Platinum Transparency 2023](https://widgets.guidestar.org/TransparencySeal/9294495)](https://www.guidestar.org/profile/46-1496217)[![SOC 2](https://www.cos.io/hs-fs/hubfs/Logos/badge-soc2.png?width=100&name=badge-soc2.png)](https://www.aicpa-cima.com/resources/landing/system-and-organization-controls-soc-suite-of-services) COS has earned top recognition from **[Charity Navigator](https://www.charitynavigator.org/ein/461496217)** and **[Candid](https://www.guidestar.org/profile/46-1496217)** (formerly GuideStar) for our financial transparency and accountability to our mission. COS and the OSF were also awarded SOC2 accreditation in 2023 after an independent assessment of our security and procedures by the **[American Institute of CPAs (AICPA)](https://www.aicpa-cima.com/resources/landing/system-and-organization-controls-soc-suite-of-services)**. We invite all of our sponsors, partners, and members of the community to learn more about how our organization operates, our impact, our financial performance, and our nonprofit status. **Photo Stream** - - - - - - - - - - - -