You may have heard that the Reproducibility Project: Cancer Biology (RP:CB) team just released results of 5 major cancer biology studies that they attempted to reproduce. This was an early step in what will be a much larger project in which over 25 studies will be revisited before the project is done--but it's already been interesting. The results were mixed and the press coverage was quite enthusiastic about the implications of that. We've put a summary page up here if you'd like to dive in.
It’s an important topic, and one that most folks in the scientific community are interested in. Here at COS, we think that these early results speak more to the research process. We are strong believers that the more open, available, and collaborative research is, the more efficient it will be. Our primary goal in undertaking this project is to find out how challenging reproducing published results is and what factors contribute to its success or failure so we can share that information with the research community.
Another goal of the project is to help clarify what the definition of "reproducible" is. The tendency is to answer the question, “Was the study replicable?” with a binary answer, “yes” or “no,” but it is much more nuanced than that. One definition of “Replicate” is to reproduce original processes and results. What is this acceptable margin of error? There are multiple, complex factors that determine this margin. Another measure of replication is subjective assessment. As part of the RP:CB, we are gathering this information through a formal survey provided by independent scientists, and will be analyzing these data at the end of the project along with other aspects of the process.
So what do these initial results tell us?
It's still really early in this project, and we will undoubtedly learn many more valuable lessons as more results come in. But the preliminary look already tells us we have plenty to work on.
Responsible stewards of your support
COS has consistently earned a Guidestar rating of Platinum for its financial transparency, the highest rating available. You can see our profile on Guidestar. COS and the OSF were also awarded SOC 2 accreditation in 2022 after an independent assessment of our security and procedures by the American Institute of CPA’s (AICPA).
We invite all of our sponsors, partners, and members of the community to learn more about how our organization operates, our impact, our financial performance, and our nonprofit status.