An often overlooked source of the “replication crisis” is the tendency to treat the replication study as a definitive verdict while ignoring the statistical uncertainty inherent in both the original and replication studies. This simplistic view fosters misleading dichotomies and erodes public trust in science.

A conference called “The promises and pitfalls of preregistration” was hosted by the Royal Society in London from 4th-5th March 2024. Here, I discuss the presentations by Chris Donkin and Stephan Lewandowsky, both of which consider some of the potential “pitfalls” of preregistration.

The inflation of Type I error rates is thought to be one of the causes of the replication crisis. Questionable research practices such as p -hacking are thought to inflate Type I error rates above their nominal level, leading to unexpectedly high levels of false positives in the literature and, consequently, unexpectedly low replication rates. In this article, I offer an alternative view.

In an article published last week in Synthese, philosopher of science Pekka Syrjänen asked “does a theory become better confirmed if it fits data that was not used in its construction versus if it was specifically designed to fit the data?” The first approach is called prediction, and the second approach is called accommodation . The debate over the epistemic advantages of prediction and accommodation has been bubbling away for

Abstract Popper’s (1983, 2002) philosophy of science has enjoyed something of a renaissance in the wake of the replication crisis, offering a philosophical basis for the ensuing science reform movement. However, adherence to Popper’s approach may also be at least partly responsible for the sense of “crisis” that has developed following multiple unexpected replication failures.

In this new article, I consider questionable research practices in the field of metascience. A questionable metascience practice (QMP) is a research practice, assumption, or perspective that’s been questioned by several commentators as being potentially problematic for metascience and/or the science reform movement. I discuss 10 QMPs that relate to criticism, replication, bias, generalization, and the characterization of science.

Researchers often distinguish between: (1) Exploratory hypothesis tests — unplanned tests of post hoc hypotheses that may be based on the current results, and (2) Confirmatory hypothesis tests — planned tests of a priori hypotheses that are independent from the current results This distinction is supposed to be useful because exploratory results are assumed to be more “tentative” and “open to bias” than confirmatory results.

In this paper (Rubin, 2021), I consider two types of Type I error probability. The Neyman-Pearson Type I error rate refers to the maximum frequency of incorrectly rejecting a null hypothesis if a test was to be repeatedly reconducted on a series of different random samples that are all drawn from the exact same null population. Hence, the Neyman-Pearson Type I error rate refers to a long run of exact replications.