A How-To Guide to Improving the Clarity and Continuity of Your Preregistration

April 26th, 2017, Alexander DeHaven


Preregistration

Preregistration, specifying your study design and analysis plan before you begin your study, has recently come to the forefront of scientific discussion as one solution to the “reproducibility crisis”. There are several benefits to preregistration: it helps “open the file drawer,” it clarifies the distinction between exploratory and confirmatory hypothesis testing, and it helps you improve your study designs. I encourage everyone to see how we are promoting more adoption of preregistration through the Preregistration Challenge and to try it out on the Open Science Framework.

During the preregistration process, you are prompted to provide several pieces of information about your research design and the analyses you plan to run. You are asked to provide at least one inferential, testable hypothesis. Other key sections of the preregistration include manipulated and measured variables and statistical models. In the variables section of the preregistration form, you simply list what variables you will be manipulating (if you are running an experiment), what you will be measuring, and how you will make those measurements. If applicable, you can also describe how your measurements will be combined into an index (e.g. using the mean average of multiple subscales included in a large questionnaire).

The statistical model portion of the form is one of the most challenging and important: here you must specify what analyses you plan to run to test the hypotheses you listed in the beginning of the preregistration. The key to this section is to take time to really think about what it is you are asking in your hypotheses and how the variables you specified could be used to best answer your research questions.

A preregistration also helps you consider how your hypotheses, variables, and analyses are worded so that readers can easily follow along. Every hypothesis contains at least one variable you wish to investigate (usually two or more!), so it is important to clearly define what this variable is and how it will be measured. If a variable in your hypotheses is abstract, then it is even more important to relate it to the measured variables you specify in the preregistration. A disconnect between your hypothesis and your variables can leave readers wondering what exactly you are testing.

Once your hypotheses and variables line up, the final step is to create your analysis plan. While doing this, you should use the same terms defined in the variables section of your preregistration. For clarity, you should go the extra mile and use the same terms and structure listed in your hypotheses in your analyses. By using these same terms throughout your preregistration, readers can easily follow your research plan and compare it to your final article. If you have more than one hypothesis, you can label each analysis plan to match each hypothesis (e.g. H1, H2, H3… and A1, A2, A3…), further improving the continuity and readability of your preregistration.

Here is one recent, public preregistration on the OSF that is a great example of consistent language, terminology, and formatting throughout the research plan. All of the measured variables and covariates appear in the final analysis plan or exploratory analysis plan. Each measured variable also appears in the hypotheses section (with the exception of “Perceived similarity of texts” as it is an exploratory measure). Also, each hypothesis has a clear counterpart in the analysis plan for the preregistration.

Ultimately, the goal of scientific publication is to disseminate information: the easier you make it on your readers to understand, the more effective and impactful your communication will be. Preregistration is a very handy tool that not only allows your research to be more transparent, but also improves the accessibility of your work. I have often grappled with the problem of making my own work more understandable when trying to explain to friends and family what I did in graduate school. Adding clarity to the research plan and preregistration will also pay dividends when it comes time to write up the final results of your work and makes the distinction between your planned and actual tests more clear.

Recent Blogs

The Content of Open Science

What Second Graders Can Teach Us About Open Science

What's Going on With Reproducibility?

Open Science and the Marketplace of Ideas

3 Things Societies Can Do to Promote Research Integrity

How to Manage and Share Your Open Data

Interview with Prereg Challenge Award Winner Dr. Allison Skinner

Next Steps for Promoting Transparency in Science

Public Goods Infrastructure for Preprints and Innovation in Scholarly Communication

A How-To Guide to Improving the Clarity and Continuity of Your Preregistration

Building a Central Service for Preprints

Three More Reasons to Take the Preregistration Challenge

The Center for Open Science is a Culture Change Technology Company

Preregistration: A Plan, Not a Prison

How can we improve diversity and inclusion in the open science movement?

OSF Fedora Integration, Aussie style!

Replicating a challenging study: it's all about sharing the details.

How Preregistration Helped Improve Our Research: An Interview with Preregistration Challenge Awardees

Some Examples of Publishing the Research That Actually Happened

Are reproducibility and open science starting to matter in tenure and promotion review?

The IRIS Replication Award and Collaboration in the Second Language Research Community

We Should Redefine Statistical Significance

Some Cool New OSF Features

How Open Source Research Tools Can Help Institutions Keep it Simple

OSF Add-ons Help You Maximize Research Data Storage and Accessibility

10 Tips for Making a Great Preregistration

Community-Driven Science: An Interview With EarthArXiv Founders Chris Jackson, Tom Narock and Bruce Caron

A Preregistration Coaching Network

Why are we working so hard to open up science? A personal story.

One Preregistration to Rule Them All?

Using the wiki just got better.

Transparent Definitions and Community Signals: Growth in the Open Science Community

We're Committed to GDPR. Here's How.

Preprints: The What, The Why, The How.

The Prereg Challenge Is Ending. What's Next?

We are Now Registering Preprint DOIs with Crossref

Using OSF in the Lab

Psychology's New Normal

How Open Commenting on Preprints Can Increase Scientific Transparency: An Interview With the Directors of PsyArxiv, SocArxiv, and Marxiv

The Landscape of Open Data Policies

Open Science is a Behavior.

Why pre-registration might be better for your career and well-being

Interview: Randy McCarthy discusses his experiences with publishing his first Registered Report

Towards minimal reporting standards for life scientists

Looking Back on the Prereg Challenge and Forward To More Credible Research

OSF: Origin, growth, and what’s next

A Critique of the Many Labs Projects

The Rise of Open Science in Psychology, A Preliminary Report

Strategy for Culture Change

New OSF Registries Enhancements Improve Efficiency and Quality of Registrations

Registered Reports and PhD’s – What? Why? How? An Interview with Chris Chambers

How to Collaborate with Industry Using Open Science

How to Build an Open Science Network in Your Community

Seven Reasons to Work Reproducibly

COS Collaborates with Case Medical Research to Support Free Video Publishing for Clinicians and Researchers

This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.