Explore preregistration

The replication crisis in science has highlighted the importance of making our research more open and transparent. Preregistration entails transparent documentation of what was planned at a certain point in time and what may have changed afterwards. This helps us to plan and track our projects, and helps others to interpret the results of our study. It also makes it possible to clearly distinguish between confirmatory analyses (based on hypotheses) and exploratory analyses (exploration of the data). As a result, preregistration can increase transparency and trust in our research.

• Hardwicke & Wagenmakers (2023) illustrate the historical evolution of preregistration, describe its functions, summarize meta-research on preregistration, and provide recommendations for using preregistration
• Nosek et al. (2018) provide arguments for preregistration and guidance for potential preregistration challenges
• Munafò et al. (2017) provide arguments for using open science techniques such as preregistration for improving the transparency, reproducibility and efficiency of scientific research

Questionable research practices (QRPs) have been blamed for the low replication rate of psychological research revealed by the landmark work of the Open Science Collaboration (OSC, 2015). Preregistration has been proposed as a remedy because it involves the publication of a study plan, which makes it impossible to covertly alter hypotheses, reduces analytical flexibility such as p-hacking, and mitigates publication bias. One way to evaluate the effectiveness of preregistration is to compare preregistered and non-preregistered research articles in terms of their outcomes. Less positive results (i.e., less confirmed hypotheses) and smaller effect sizes in preregistered studies would indicate that preregistration counteracts QRPs present in non-preregistered studies. The listed studies followed this approach.

• Schäfer & Schwarz (2019) found a lower proportion of positive results (0.64 vs. 0.79) and lower median effect sizes (0.16 vs. 0.36) in preregistered vs. non-preregistered publications.
• Scheel et al. (2021) found a lower proportion of positive results in published Registered Reports than in non-preregistered publications (0.44 vs. 0.96).
• Toth et al. (2021) also found a lower proportion of positive results in preregistered than in non-preregistered studies (0.48 vs. 0.66).
• Van den Akker et al. (2023) also compared preregistered vs. non-preregistered studies and did not find that preregistered studies had a lower proportion of positive results (but they report other benefits).

A second approach to examine the effectiveness of preregistration is by estimating its ability to limit analytical flexibility. Some studies also compared preregistrations with the resulting publications to determine whether deviations occurred and, if so, whether they were reported transparently. In conclusion, the studies argue for detailed, structured preregistration templates and transparent reporting of deviations, as they all found room for improvement. In line with these findings, we provide structured and detailed templates for creating both preregistration and deviation protocols.

• Bakker et al. (2020) compared preregistrations created from scratch vs. with structured templates. The structured format restricted the opportunistic use of researcher degrees of freedom better.
• Van Den Akker et al. (2023) confirmed this result in a larger sample.
• Heirene et al. (2021) used scoring protocols to evaluate the restrictiveness of preregistrations and their congruence with the resulting publications. Since the scores were rather low, they developed practical recommendations on what should be improved.

Meta-scientific studies, such as those listed, show that preregistration and other open science techniques are becoming increasingly recognized and utilized within the psychological research community. Correspondingly, many researchers report that they have already preregistered or intend to preregister in the future, motivated by the desire for greater transparency, trustworthiness, and improved planning. At the same time, obstacles were identified that currently hinder preregistration, such as effort and time, lack of knowledge and uncertainty about how to deal with deviations. Addressing these challenges through increased support and workshops could facilitate broader adoption of preregistration in the future.

• Stürmer et al (2017) surveyed early career researchers regarding the prevalence of questionable research practices and open science as a solution
• Abele-Brehm et al. (2019) inspected German researchers’ fears and hopes regarding open science and data sharing.
• Spitzer & Mueller (2023) conducted a survey among psychological researchers regarding attitudes, motivations, and obstacles of preregistration, and discuss potential solutions for the latter