The Reproducibility Project: Cancer Biology was an 8-year effort to replicate experiments from high-impact cancer biology papers published between 2010 and 2012. The project was a collaboration between the Center of Open Science and Science Exchange with all papers published as part of this project available in a collection at eLife and all replication data, code, and digital materials for the project available in a collection on OSF.
When preparing replications of 193 experiments from 53 papers there were a number of challenges.
experiments with open data
of experiments required asking for key reagents
of experiments needing a key reagent original authors were willing to share
of protocols completely described
of experiments the original authors were not helpful (or unresponsive)
of experiments the original authors were very helpful
Fully designed protocols were submitted to eLife for peer review before conducting the replications - a publishing format called Registered Reports - to ensure that the proposed experiments were of the appropriate rigor and quality. Accepted replication protocols received a commitment in advance to publish the findings regardless of outcome.
Additional challenges were encountered for the experiments that were conducted.
required modifications to complete
of modifications completely implemented
Ultimately, 50 replication experiments from 23 of the original papers were completed, generating data about the replicability of a total of 158 effects. There are many ways to evaluate and characterize replication outcomes, some simplified summaries of the findings include:
Collectively, this evidence suggests opportunities to improve the transparency, sharing, and rigor of preclinical research to advance the pace of discovery.