Symposium: Critical Perspectives on the Metascience Reform Movement

March 7, 2024 | 9:00 am - 12:30 pm EST

Watch the Recording


Metascience is often defined as the scientific investigation of science itself with the aim to improve science. This ‘improving’ part of metascience has been called a reform movement. While the intention to improve science is generally laudable, metascience and the associated movement(s) are not without their critics. The metascience reform movement has for instance been characterized as among other things, neoliberal, positivistic, atheoretical, technological, moralizing, bureaucratic, homogenizing, mechanistic, quantitative, psychological (psychologizing), social, civilizing, bullying, exclusive, coercive, activist, and normative. In this symposium some of those critical perspectives will be presented and discussed.

Session Recordings


Please find the webinar recording below if you couldn’t attend the event or would like to re-rewatch any session or send it to interested colleagues.




Presenters will provide a 20-minute perspective followed by a concluding panel discussion and a time to address audience questions. The symposium will conclude with a 40-minute panel discussion and Q&A.

two speakers

Tim Errington (left), Center for Open Science, and Sven Ulpts (right),
Aarhus University



6:00 am PST / 9:00 am EST / 3:00 pm CET

Opening remarks from Tim Errington, COS's Senior Director of Research, and Sven Ulpts, PhD Fellow, Aarhus University, Department of Political Science - Danish Centre for Studies in Research and Research Policy.

Bart Penders

Bart Penders
RWTH Aachen University

Shamed Into Good Science

6:10 am PST / 9:10 am EST / 3:10 pm CET

Scientific reform has proposed various improvements to the research process, some of which have been put into place. These proposals, or innovations, include preregistration, registered reports, post-registration, (data) sharing infrastructures and -platforms and more. However, beyond the ‘process innovations’, scientific reform also generated new forms of collaboration towards knowledge-making in which novel approaches to harnessing trust and distrust are built into the collaboration, including but not limited to ManyLabs approaches. The many “ManyLabs” in existence display diverse and experimental epistemic and moral strategies and ideals for scientific organisation and practice. I will speak about the social processes that drive these new processes and colaborations.

Thomas Hostler

Thomas Hostler
Manchester Metropolitan University

The Invisible Workload of Open Research

6:35 am PST / 9:35 am EST / 3:35 pm CET

It is acknowledged that conducting open research requires additional time and effort compared to conducting ‘closed’ research. However, this additional work is often discussed only in abstract terms, a discourse which ignores the practicalities of how researchers are expected to find the time to engage with these practices in the context of their broader role as multifaceted academics. There is a clear danger that additional expectations to engage in open practices add to the workload of often already overburdened academics and research professionals. Using theories of administrative burden and workload creep, I discuss individual and systematic responsibilities to mitigate this.

Stephan Guttinger

Stephan Guttinger
University of Exeter

Why Replication Is Underrated

7:00 am PST / 10:00 am EST / 4:00 pm CET

I will argue that replication is underrated because it can do more epistemic work than is usually acknowledged. In the current philosophical and metascientific literature replication is depicted as a backward-looking tool: it is used to increase trust in published data. I will argue that replication also plays an important forward-looking role, as it can help researchers to build trust in novel data that is being generated. To understand this forward-looking role it is important to build a better understanding of the ways in which replication is deeply integrated into everyday research practice (Guttinger 2019; Peterson and Panofsky 2021).

Sarahanne Field

Sarahanne Field
University of Groningen

Communities of Practice in the Science Reform Movement - Recognizing and Mobilizing Diverse Contributions to Science Reform

7:30 am PST / 10:30 am EST / 4:30 pm CET

When people write and talk about science reform, they refer to a movement, or a single community of practice centered on improving how science is done and shared. An ethnography and subsequent network analysis I conducted on the online part of the movement reveals evidence of a much more complex research and activism landscape, however. My work indicated that while there indeed exists a singular purpose, or 'joint enterprise' which unites much of the science reform's activity, the group of people driving the movement tend to organise into smaller clusters within the broader landscape. They group together based on different priorities, methodologies and even philosophies of science; a constellation of small communities of practice. My talk presents this preliminary research, and outlines some interesting implications it might have for considering the future of the reform movement.

Nicole C. Nelson

Nicole C. Nelson
University of Wisconsin-Madison


The Decline of Scientific Individualism?

7:55 am PST / 10:55 am EST / 4:55 pm CET

A number of metascience innovations—such as the Psych Science Accelerator, Many Analysts projects, the Multi-PART study, and cloud labs—involve an explicit move away from individuals or small teams as the basic unit of scientific research. This talk will ask whether metascience as a movement might be hastening the decline of scientific individualism (if such a decline is indeed taking place!), and what epistemic and social risks might be associated with these changing work practices.

Berna Devezer

Berna Devezer
University of Idaho


Meta Makeover: Ditching Dogmas and Raising Standards of Evidence

8:20 am PST / 11:20 am EST / 5:20 pm CET

Metascience and scientific reform have initiated a social movement toward procedural changes in scientific practice. Prioritization of prescriptive recommendations (e.g., preregistration, replication) over theoretical understanding or evidence-based solutions has led to uncritical, even dogmatic interpretations of reforms. Evidential standards are relaxed when evaluating metascientific results that endorse reforms. Self-serving and self-congratulatory narratives have led to premature metascience hubris. Without intellectual humility, however, there is no hope for meaningful progress. I suggest it is time to course correct, time for (meta)scientists to ditch dogmas and heuristics, and refocus on science.

Sign up to join researchers worldwide for Critical Perspectives on the Metascience Reform Movement on March 7, 2024.


Thank You to Our Event Co-Sponsors