Like Upgrading From a Typewriter to a Computer - Registered Reports in Education Research

Oct 15, 2026

Upload by Cart

“Like Upgrading From a Typewriter to a Computer”: Registered Reports in Education Research explains how registered reports change journal evaluation by reviewing the research question and prespecified design before results are known. The paper outlines Phase I submission and “in-principle acceptance,” then describes how adherence to the original design leads to later acceptance. It argues that the format strengthens transparency, reduces incentives for questionable practices, and helps limit illusory findings in education research.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages6

Reading Time15 min

AccessFree to download

Registered reports format

Phase I submission and in-principle acceptance

Benefits for research integrity

Early review and reduced incentives

AERA Open April-June 2020, Vol. 6, No. 2, pp. 1–6 DOI: 10. 1177/2332858420917640 Article reuse guidelines: [sagepub.com/journals-permissions](sagepub.com/journals-permissions)[ ](sagepub.com/journals-permissions)© The Author(s) 2020. [http://journals.sagepub.com/home/ero](http://journals.sagepub.com/home/ero) “Like Upgrading From a Typewriter to a Computer”: Registered Reports in Education Research Justin Reich Massachusetts Institute of Technology Hunter Gehlbach Johns Hopkins University Casper J. Albers University of Groningen Registered reports engender a review process in which reviewers evaluate the merits of the research question rather than the magnitude or direction of the results. In this process, authors first submit introduction, methods, and prespecified analyses sections of an article. Next, an initial peer-review process assesses the merits of the research question, in the absence of knowing the results. Worthy articles are granted an “in-principle acceptance.” The authors, then conduct their study, analyze their data, write up the remainder of their article, and resubmit it. The in-principle acceptance means that if the authors adhere to the research design as originally submitted, then the article will be accepted. Building from seven of the earliest registered reports in education research, our introduction describes the benefits, opportunities, and challenges of registered reports as a means to increasing transparency in research and reducing the number of illusory results. Keywords: evaluation, open science, preregistration, registered reports, research methodology, research utilization, standard setting Recent observers ofthe scientific process have called attention to a set of related issues that threaten the integrity of the scientific literature (Ioannidis, 2005; John et al., 2012; Van der Zee & Reich, 2018) . These issues range from a lack of replication studies to “p-hacking” to a lack of transparency in analytic decision making (Gelman & Loken, 2014; Makel & Plucker, 2014; Simmons et al., 2011) . In an ideal world, research that tackles compelling scientific questions and urgent public problems with the highest quality methodological approaches would be the most “publishable”research. Instead, scientific research has disproportionately been judged on the direction and magnitude of findings, or novelty of results, rather than on methodological rigor (Makel & Plucker, 2014) . As a result, in many social science fields, illusory results buttressed by elaborate stories have saturated the research literature; null results are underrepresented. Gehlbach and Robinson (2018) liken the current state of science to the steroids eras in baseball. Performance-enhancing techniques in both have caused crises of confidence and credibility. One of the most promising innovations in addressing these issues is the focus of this AERA Open special topic: Registered Reports. Registered reports are a new format of journal article where authors first submit a “Phase I” manuscript that includes the introduction, background and context, and methods (including prespecified analyses) sections of an article (Chambers, 2013; Chambers et al., 2015) . Peer reviewers evaluate the importance of the research questions and the appropriateness of the data, method, and analytic approach, in the absence of knowing what the results might be. At this point editors can then grant the article “in-principle acceptance,” request revisions, or even suggest modifications of the study design. An in-principle acceptance means that if the authors adhere to the research design as described in the Phase I manuscript, then the article will be accepted for publication regardless of the direction or magnitude of findings. After an in-principle acceptance, the authors then conduct their study or analyze their data, finish writing up their article, and submit a final draft that is then reviewed by the editor (and reviewers as need

FAQs

What is the core idea behind registered reports in education research?

Registered reports require reviewers to evaluate the research question and prespecified design before results are known, focusing on methodological merit rather than the direction or magnitude of findings.

How does the “in-principle acceptance” mechanism work?

Editors can grant “in-principle acceptance” after Phase I review. If authors follow the Phase I research design as submitted, the article is accepted regardless of whether findings are positive or negative.

What problems in scientific literature does the paper aim to address?

It targets issues such as weak transparency in analytic decisions, limited replication, and practices that produce illusory or misleading results, including incentives that favor “significant” outcomes.

Oct 15, 2026

Upload by Cart