Rogue Scholar Posts

language
Published in Critical Metascience
Author Mark Rubin

The inflation of Type I error rates is thought to be one of the causes of the replication crisis. Questionable research practices such as p-hacking are thought to inflate Type I error rates above their nominal level, leading to unexpectedly high levels of false positives in the literature and, consequently, unexpectedly low replication rates. In this article, I offer an alternative view.

Published in Critical Metascience
Author Mark Rubin

I contrast Popper's (1983, 2002) theory testing approach with that of Lakatos (1978) and a related approach called naïve methodological falsificationism. I conclude that the replication crisis is least problematic in the Lakatosian approach.

Published in Stories by Mark Rubin on Medium
Author Mark Rubin

In this new article, I consider questionable research practices in the field of metascience. A questionable metascience practice (QMP) is a research practice, assumption, or perspective that’s been questioned by several commentators as being potentially problematic for metascience and/or the science reform movement. I discuss 10 QMPs that relate to criticism, replication, bias, generalization, and the characterization of science.

Published in Stories by Mark Rubin on Medium
Author Mark Rubin

Researchers often distinguish between: (1) Exploratory hypothesis tests — unplanned tests of post hoc hypotheses that may be based on the current results, and (2) Confirmatory hypothesis tests — planned tests of a priori hypotheses that are independent from the current results This distinction is supposed to be useful because exploratory results are assumed to be more “tentative” and “open to bias” than confirmatory results.

Published in Stories by Mark Rubin on Medium
Author Mark Rubin

In this paper (Rubin, 2021), I consider two types of Type I error probability. The Neyman-Pearson Type I error rate refers to the maximum frequency of incorrectly rejecting a null hypothesis if a test was to be repeatedly reconducted on a series of different random samples that are all drawn from the exact same null population. Hence, the Neyman-Pearson Type I error rate refers to a long run of exact replications.