What is it about?
Preregistration entails researchers registering their planned research hypotheses, methods, and analyses in a time-stamped document before they undertake their data collection and analyses. This document is then made available with the published research report in order to allow readers to identify discrepancies between what the researchers originally planned to do and what they actually ended up doing. In a recent article (Rubin, 2020), I question whether this "historical transparency" facilitates judgments of credibility over and above what I call the "contemporary transparency" that is provided by (a) clear rationales for current hypotheses and analytical approaches, (b) public access to research data, materials, and code, and (c) demonstrations of the robustness of research conclusions to alternative interpretations and analytical approaches.
Featured Image
Photo by Scott Graham on Unsplash
Why is it important?
There are over 18,000 study preregistrations on the Open Science Framework, and the annual additions are doubling every year (Kupferschmidt, 2018). Given the speed of this uptake, it is important to critically analyse the potential added value of preregistration's historical transparency over and above contemporary transparency. My article covers issues such as HARKing, multiple testing, p-hacking, forking paths, exploratory analyses, optional stopping, researchers’ biases, selective reporting, test severity, publication bias, and replication rates. I argue for a nuanced approach to these issues. In particular, I argue that only some of these issues are problematic, and only under some conditions. I also argue that, when they are problematic, these issues can be identified via contemporary transparency per se.
Perspectives
I conclude that preregistration’s historical transparency does not facilitate judgments about the credibility of research findings when researchers provide contemporary transparency. Of course, in many cases, researchers do not provide a sufficient degree of contemporary transparency (e.g., no open research data or materials), and in these cases preregistration’s historical transparency may provide some useful information. However, I argue that historical transparency is a relatively narrow, researcher-centric form of transparency because it focuses attention on the predictions made by specific researchers at a specific point in time. In contrast, contemporary transparency allows research data to be considered from multiple, unplanned, theoretical and analytical perspectives while maintaining a high degree of research credibility. Hence, I suggest that the open science movement should push more towards contemporary transparency and less towards historical transparency.
Prof Mark Rubin
Durham University
Read the Original
This page is a summary of: Does preregistration improve the credibility of research findings?, The Quantitative Methods for Psychology, May 2020, The Quantitative Methods for Psychology,
DOI: 10.20982/tqmp.16.4.p376.
You can read the full text:
Resources
Contributors
The following have contributed to this page
