Introducing the Theory-Based Predictions in Social Science Preregistration Template: Q&A with Andrew Cesare Miller

Social scientists routinely use theory to make predictions about future events—from electoral outcomes to conflict dynamics to policy effects. These predictions help shape public debate and inform real world decision-making. Yet unlike explanatory claims about past events, predictions are rarely made in ways that allow them to be systematically evaluated over time. Without clearly defined outcomes, timeframes, and scope conditions, many theoretical predictions remain difficult—or impossible—to falsify.

A growing line of work has begun to address this gap by extending preregistration beyond experiments and observational studies to theory-based predictions themselves. This approach, termed prediction registration, was first articulated by Andrew Cesare Miller in “Registering Theory-Based Predictions in Political Science”, published in PS: Political Science & Politics. In the article, Miller argues that “increasing the rigor of predictive theory testing can advance often-circular debates about accuracy and presents a ‘win-win’ for scholars who aim to test the predictive power of theories.”

Building on this framework, Miller developed the Theory-Based Predictions in Social Science preregistration template, now available on the Open Science Framework (OSF). The template guides researchers in specifying falsifiable predictions by documenting theories, variables, measurable outcomes, time horizons, and scope conditions—creating a transparent record that allows predictions to be evaluated after publication.

An Assistant Professor of Political Science at the United States Naval Academy*, Miller’s research explores the intersection of politics, policing, and organized crime. In this Q&A, he discusses what motivated the development of the template, how it differs from conventional preregistration approaches, and how registering predictions could change how social scientists assess theoretical claims and cumulative evidence over time.

Q: What motivated you to develop a preregistration template specifically for theory-based predictions in social science?

A: Social scientists often use theories to make predictions about future outcomes, both in peer-reviewed outlets and in public commentary. They do so for good reason. Accurate predictions can be extraordinarily valuable to policymakers. Indeed, almost all policy decisions hinge on an implicit prediction about what will unfold as a result of a given decision (e.g., “if we implement X policy, Y outcome is likely to occur”). Policy and prediction are, as political scientist Kristian Skrede Gleditsch puts it, “like love and marriage.”

Yet, it is unclear how, if at all, accurate the theories used to make predictions are, because the rigor employed for testing claims of theories' predictive power lags well behind that for testing claims of explanatory power about past events. Predictions often are made in an unfalsifiable manner in that there is no way to evaluate if a given prediction was accurate and thus no way to evaluate whether the theory used has predictive power. The goal of the prediction registration template, accordingly, is to specify parameters required to preregister falsifiable predictions, including time frames, measurable outcomes, independent variables, and scope conditions.

Q: How does this template differ from existing templates designed for experiments or observational studies?

A: This template lays out the parameters required to engage in the “prediction registration” process. Conventional study registration typically involves a researcher predicting an outcome before data collection, collecting the data, analyzing the results, and then publishing findings. However, many social science theories relate to events that are predicted to occur within wide temporal windows –my analysis of predictions published in top international relations journals found that, on average, scholars predicted events that would occur eight years after an articles’ publication date with one article even including a prediction of an event 78 years in the future. Accordingly, prediction registration calls for scholars – and editors – to register and publish predictions even if the outcome is not yet known. Published articles link to the study’s registration page, so that anyone can evaluate the accuracy of the predictions post-publication.

Q: What are common challenges with how predictions are currently made and evaluated in the social sciences? How does this template address those challenges?

A: As noted above, many predictions made using social science theories are unfalsifiable. My analysis of predictions in international relations journals found that only about 1% of all articles with a prediction contained what could be considered at least one falsifiable prediction. The template attempts to address this challenge by having researchers specify the parameters required for a prediction to be falsifiable – researchers must document their theory, specify variables, define outcomes, and set timeframes for when the event is expected to (not) occur. 

Registering predictions with the template has two additional benefits. For one, this process facilitates systematic aggregation of predictions. Predictions using a given theory are collected on study registration sites with the same set of parameters as specified in the template and thus the predictions can be evaluated more as a body of evidence rather than isolated claims. Furthermore, the registration of predictions provides a transparent record (including logging any updates to the predictions) to ensure external parties can verify prediction accuracy. 

Q: What impact do you hope the Theory-Based Predictions in Social Science template will have on how political scientists and other social scientists test theories and evaluate predictive claims?

A: I hope that we see more and more falsifiable predictions on social science topics. There arguably are professional incentives against making predictions; given the difficulty of prediction, making public, falsifiable predictions might be seen as too great a risk to the credibility of one’s theory. However, registering predictions is also arguably a “win-win” for researchers: either their predictions prove correct, or incorrect predictions reveal paths to improve the theory by identifying new variables, more precise scope conditions, and other factors. 

I also hope that journal editors will increasingly encourage authors who claim a theory to have predictive power to demonstrate that they have registered their predictions. Procedurally, this would not be a radical departure from existing editorial processes – it would simply expand what has become known as the “preregistration revolution” that has already made registration the standard for many disciplines and subfields in social science. Previously, few scholars used registration for experimental studies but now registration has become a standard expectation of studies using experimental methods. Registrations to test the predictive power of theories can follow suit. 

Q: Is there anything else you would like to add about the template, the development process, or your hopes for how the research community will adopt and adapt it?

A: Sometimes it is argued that human behaviors are simply too difficult to predict, which implies that registering predictions to test and iteratively improve theories would be a Sisyphean endeavor. There is a growing body of evidence, however, that improving predictive accuracy about human behavior is possible. One study, for instance, found that the U.S. government increased their originally highly-accurate predictions about nuclear proliferation, eventually achieving correct assessments in 80% of cases. An evaluation of U.S. presidential-election predictions, for its part, found that they have become increasingly accurate over time. My hope is that this template will accelerate such improvements.

Access the Theory-Based Predictions in Social Science Template

Preview this template here.

Interested in using this template? Get started by creating a new preregistration on OSF.

Propose A New Template

In 2023, COS launched a working group to help curate, evaluate, and promote high-quality preregistration templates that reflect the needs of different research communities on the Open Science Framework (OSF). As part of this ongoing effort, COS invites OSF users to propose new preregistration templates through this submission form.

*The views expressed in this interview do not represent those of the Naval Academy or U.S. government.