Lifecycle Open Science in Action: Researcher Q&A with Andres Colubri

Research doesn't begin with a published paper—it starts with a question, a plan, and a process that sometimes remains invisible to others. At the Center for Open Science (COS), we use the term lifecycle open science (LOS) to describe an approach to making the record of your work visible: research with publicly accessible plans, outputs (such as data, materials, and code), and outcomes that are linked and findable across the research lifecycle.

Lifecycle open science begins with a prospective study plan and preregistration to set a clear, time-stamped foundation for your research. For researcher Andres Colubri, MFA, PhD, preregistration is less a formality than a working record of how a hypothesis actually develops. As he put it, it's a kind of "seal of scientific reproducibility" that researchers themselves create by stating in advance what they propose to do and how, and leaving it open for others to inspect. That record becomes especially valuable when the original hypothesis doesn't hold: rather than treating that as a dead end, Colubri sees it as the point where refinement happens, with the preregistration then updated to document the shift.

Colubri is an Assistant Professor at UMass Chan Medical School, where his multidisciplinary lab builds digital platforms to study infectious disease transmission. His study, An Experimental Epidemic Game (Epigame) to Model the Attitude-Behavior Relationship in Prophylactic Quarantine, uses a phone-based game with Bluetooth proximity sensing to simulate an outbreak among real participants, testing whether increasing the in-game cost of adopting voluntary quarantine changes how readily people choose to protect themselves. The study is registered on the OSF and includes a preregistration update written after the data led the team to reject their original hypothesis and refine it.

The project is a concrete illustration of lifecycle open science in practice: Colubri sees the paper as just one part of the research record and views the preregistration, its update, and the connected research materials as complementary pieces of that record—together showing not just what the study found, but how the thinking behind it changed along the way.

In this Q&A, Colubri delves into the design of the Epigame study, why he updated the preregistration after the data led the research team to reject their original hypothesis, and what he sees as the value of keeping a project's plans, outputs, and outcomes connected and open.


Q: Can you give an overview of the Epigame project—what were you and your collaborators hoping to understand, and what led you to model it as a game?

A: Epigame is a shorthand for Epidemic Game. The whole term was originally Experimental Epidemic Games. I was trying to expand on an existing approach called Experimental Games, which has been used for a long time to study how people make decisions in complex scenarios. I inserted the word epidemic because these games focus on epidemiological scenarios.

Essentially, these are real-world games where participants experience epidemiological scenarios—for example, the transmission of a hypothetical pathogen—as part of their daily routines. Our current implementation uses a mobile app with Bluetooth proximity sensing, so we know who is near whom and can simulate how a disease spreads from an initial index case through the player population.
As they play, they make decisions in the game. For example, I'm going to protect myself by choosing to quarantine. There’s a cost associated with each decision, measured with a point score. An epigame can introduce different gamified incentives, such as a lottery prize that require a high final score, but without requiring players to follow a predetermined strategy to make points.

What we try to do is model the cost-benefit associated with different preventive behaviors or interventions in real-life. For instance, if you adopt voluntary quarantine because of an infectious disease outbreak, you lose the possibility of socializing, going to work, etc. These restrictions impose a cost in your life, which we implement as an immediate reduction in score during the game. At the same time, quarantine protects you from the possibility of infection, which in real-life can have a serious health cost later. This could be represented in the game by a much larger score reduction upon infection. This balance is what we try to gamify.

Our larger goal is to use games as an experimental tool to understand why people make decisions in response to various health risks. We focus on infectious diseases, but the idea of Epigames could be applied to other scenarios; for example, addictions, mental health, or chronic diseases.

The main purpose of this particular study we registered on OSF, was first of all to measure the beliefs that people have with regard to outbreaks in real life, and similar beliefs in response to the game scenario. Then the key question is: do these beliefs correlate or not? We don’t expect to find a perfect correlation, but at least whether there is some correlation or not. That was one of the questions that we preregistered.

This is a randomized control trial study, where, for some people, there was a baseline cost to adopt quarantine. For the second group, the cost was much higher. We wanted to see if that difference in the economic cost of quarantine made a difference in the adoption of quarantine.

There are many reasons why we use this idea of experimental games. It's a methodology inspired by behavioral economics and game theory. There are games around public goods, or trust games that have this element of economic incentives. People in behavioral economics have been using it for many years to understand what factors have an influence on decision-making.

Second, there are interesting examples of people studying infectious diseases using games. There's a famous example going back more than 20 years, in an actual multiplayer game called World of Warcraft. The game, in principle, has nothing to do with infectious diseases, but there was an accidental outbreak due to a bug in the game called the Corrupted Blood Incident. The bug resulted in a pandemic happening within World of Warcraft, and people behaved in surprisingly realistic ways. Epidemiologists took this as an indication that games could be used as petri dishes for studying human behavior in a safe setting and observing how people behave in response to a simulated health risk.

That's what we want to study more broadly: what happens in the game, and to what extent it maps to real-life behaviors. If we can prove there’s some degree of mapping, then games could be a useful tool to understand human behavior and how people make decisions in response to various health risks, which could have implications for real-life policymaking.

Finally, an important inspiration for Epigame comes from my work in a project that I co-created in 2018 with collaborators at the Broad Institute of Harvard and MIT called Operation Outbreak, which focused on education around infectious diseases. Those core ideas—proximity sensing and simulating a disease spreading through the players—inspired the work I've been doing with Epigames, which is focused on using these techniques for socio-behavioral and network-science research.


Q: Your preregistration includes a formal update documenting what changed between the first and second round of the experiment, and why. What led you to make this update, and what do you see as the broader value of preregistration for how you plan and carry out your research?

A: What led me to the update is that the hypothesis I preregistered in the original version of the study was rejected. I essentially proposed a number of hypotheses. You have to do some extra work of going through the OSF interface, entering all the information—but it’s just going through some forms. I think it really helps, because you want to make this visible and readable by others, so you put some extra effort to make it clear. It also helps clarify things to yourself. That's all part of why I think registration is important.

Having an initial preregistered study sets the foundation of the work: this is the hypothesis, this is how we're going to analyze the data—and we just did that. Because the original hypothesis was rejected and we observed other, more subtle patterns in the data, it led to potential new hypotheses. So it's this process of: I have an initial hypothesis, I have data, I reject or accept the hypothesis, then I refine it. Here, the hypotheses as they were presented initially were not correct, but we saw more patterns in the data that led to a refined version of the hypothesis. The revision has that refined version.

For the original version that is preregistered, we ran a study in November, got the data, and because everything was already preregistered, including the methods, it was quite easy to  go ahead with what we wanted to do. We just followed the protocol in the way it was preregistered. That really helped with the whole process because we thought through everything beforehand. Some minor details changed a bit from the original protocol to the actual implementation, but for the most part, we did what we preregistered originally.

Then we updated the hypothesis because we rejected the original one, went back, updated the preregistration, and arrived at this new hypothesis. We ran another study in May and already have the data, so we're now running the analysis.

What we did is essentially how science operates: we had a set of initial hypotheses that were not correct, but we were able to find patterns in the data that led us to a new set of hypotheses to be tested with a new experiment. All of this is clearly documented through the update in our preregistration and method, and we went according to that publicly-available plan. It allowed us to document this whole iterative process. I think it's very helpful because I can always refer back to it, see the process, and share it.


Q: Your project connects a preregistration, preprint, and research materials in one place. What do you see as the benefits of keeping these elements connected and publicly accessible throughout the research lifecycle?

A: There's a personal convenience, because I have access to everything in one place. When you are working on many projects, you start having files here and files there—projects pile up, one project leads to another, then a derived project. I see a lot of value in putting everything into a package that makes it very easy, even just for myself, to know it's there—and if I want to share this with someone else, I can just say, “Here's a link, this has everything you need.” 

In all my manuscripts related to this project, I put the preregistration link, and I think that becomes a really important part of the whole research output. We tend to think of the paper as the main output, but to me the paper is just one part of it. All of these other elements are very important, and so is having them all together.


Q: When all elements of a project are linked and openly available, how do you think that affects other researchers' ability to trust, evaluate, or build on the work?

A: To me, it's like a seal of scientific reproducibility to have these preregistrations. And it's not a seal that comes from OSF—it's us, the scientists, saying: this is what we propose to do, this is how we're going to do it, and it's open for everybody to inspect.

That's what we try to do in our subsequent work, to the best of our ability. Having worked in open-source projects in the past, I think this creates trust in a couple of ways. One is really about contributing to the community—these are my research protocols, how I want to do it, and it's there for anybody to use. The other is that it helps create trust. It gives people who have open science, accessibility, and reproducibility as guiding principles a framework to actually implement and realize those principles.

Research involves so many things you need to take care of, and documenting your protocol and making it available is all extra work. Sometimes it falls by the wayside, because other things feel more pressing—you want to get the study done and published. So you know it's important, but you don't always have time. Having a platform that facilitates that is very important because it incentivizes people to adopt these best practices. There's additional work in going through the preregistration process and writing it in a format that's clear to others, but it really pays off. It's something I use during the subsequent steps, because I've already gone through thinking of how to do it, and it's clearly spelled out. That initial time investment helps during the actual analysis stages.


Q: What advice would you give to researchers who are just beginning their open science journey—particularly around preregistration and making their work findable and connected?

A: I think science, by definition, is an open endeavor, because in order to do science, we need to build on the contributions of others. The whole idea of publishing research papers is one implementation of that, because papers are meant to provide all the information that others might need to reproduce the work. But papers are written after the study has been completed.

Research preregistrations and manuscripts both have critical important roles in open science, since it's entirely based on exchange, contribution, and collaboration. If it’s proprietary research that happens in industry, maybe it's different and one could speak of “closed science” in that context, but here we're talking about basic research that we do as part of a larger community to advance knowledge more broadly. So it needs to be open by definition.

Manuscripts give you only one part of the equation to realize open science. Yes, you publish your methods and results—but having documentation of where things start, before conducting the research, is very important. I also think the whole process needs to be documented, because publications tend to focus on just positive results. By documenting your hypothesis for the community, you can have a more complete picture of how research proceeds.

In the case of my own work on Epigames, we created an initial registration, we refined the registered hypothesis based on the data, and then we made an update to the registration—you can see the whole process. That process of iterating and refining the hypothesis through preregistration and subsequent updates is very important to understanding how we’re gaining more insight about human decision making using Epigames.