Is a 55% Replication Rate Too Low, Too High, or Just Right?

Tyner et al. (2026) recently reported the results of a major study investigating the replicability of claims in the social and behavioural sciences. They found a replication rate of 55.1% of 274 claims. But is this replication rate too low, too high, or just right?

They concluded that their findings are consistent with prior work that has found "low observed replication success rates" (Tyner et al., 2026, p. 143). But how do we know that an observed replication rate is "low"? As I've explained previously, "it is unclear how replication rates can be judged to be 'low' and in need of improvement in the absence of clear targets for 'acceptable' replication rates. Logically, this reasoning represents an incomplete comparison" (Rubin, 2023, p. 4). We need to consider relevant benchmarks against which to judge observed replication rates. Here, I consider two possibilities: the optimal replication rate and the expected replication rate.

The Optimal Replication Rate

One benchmark that might be used to establish whether an observed replication rate is "low" is the optimal replication rate. However, defining an optimal replication rate is problematic because it leads us to ask: "optimal for what purposes?" and different replication rates may be optimal for different purposes. To illustrate, I consider how different stages of research and different philosophies of science might warrant different optimal replication rates.

Stage of Research

The optimal replication rate needs to balance the desire to learn from our mistakes in the face of scientific ignorance with the desire to produce reliable and trustworthy scientific knowledge that can be used by society. This balance is often skewed more towards learning in the earlier, discovery and exploration stages of a research topic and more towards consolidation and verification in later, more applied and translational stages. As Nosek et al. (2022) explained:

It would be possible to achieve near-100% replicability by adopting an extremely conservative research agenda that studies phenomena that are already well understood or have extremely high prior odds. Such an approach would produce nearly zero research progress. Science exists to expand the boundaries of knowledge. In this pursuit, false starts and promising leads that turn out to be dead ends are inevitable. The occurrence of nonreplicability should decline with the maturation of a research topic, but a healthy, theoretically generative research enterprise will include nonreplicable findings. (p. 730)

Hence, optimal replication rates are likely to be lower in the earlier exploratory stages of a research topic than in the later confirmatory stages that are intended to translate findings into real-world applications.

Philosophy of Science

The tension between learning and confirming is also apparent in different philosophies of science. Hence, the optimal replication rate may also be lower in theory-centric falsificationist philosophies of science than in effect-centric confirmatory philosophies. In falsificationist philosophies, (reproducible) replication failures may entail logical refutations of theories that fuel scientific discovery and progress (Popper, 1966, p. 285; Rubin, 2025, p. 11). As Popper (1962) explained, "refutations have often been regarded as establishing the failure of a scientist, or at least of [their] theory. It should be stressed that this is an inductivist error. Every refutation should be regarded as a great success" (p. 243). Accordingly, high quality, reproducible replication failures should be seen as causes for celebration rather than grounds for a replication "crisis." As Mayrhofer et al. (2024) explained:

From this [Popperian] perspective, the replication crisis is not a crisis at all but rather a process that increases our knowledge by demonstrating that certain theories are false or at least cannot be corroborated by repeated observations, increasing their probability of being false. (p. 4)

Similarly, from a Lakatosian perspective, replication failures may inspire theory development by suggesting the presence of unrecognized moderators and boundary conditions (e.g., Lakatos, 1978; Rubin, 2025). As Nosek (quoted in Jones, 2026) explained:

Replication, when it fails, is theoretically generative. It's like: "Wait a second, did we change something that we didn't realize was important? We had no reason to expect something different here, but something different happened. Now we have a mystery." And that's where discovery happens. (p. 38)

In contrast, effect-centric confirmatory philosophies focus more on establishing the existence of phenomena vis-à-vis their replicability before proceeding to testing theories that explain those phenomena. As Nak et al. (2026) explained:

The massive replication work that has ensued in the wake of the reproducibility crisis (Nosek et al., 2022) in many ways already exemplifies this turn towards phenomena, as it clearly was never aimed at theory testing but simply at assessing the replicability of psychological effects. For instance, the Open Science Collaboration (2015) set out to estimate the replicability of "effects" rather than to confirm or falsify theories. (p. 24)

From this effect-centric perspective, replication failures cast doubt on the existence of phenomena. Hence, the optimal replication rate should be higher under a confirmatory philosophy of "phenomena consolidation" (Nak et al., 2026) than under a falsificationist philosophy of theory development (Lakatos, 1978; Popper, 1962, 1966). (See also Devezer & Buzbas', 2023, distinction between "result-centric" and "model-centric" science and Feest's, 2024, distinction between "effect-seekers" and "complexity mongers.")

In summary, the optimal replication rate may vary depending on both (a) stage of research and (b) philosophy of science. In particular, optimal replication rates should be lower in discovery and exploratory stages of research and research that follows a theory-centric falsificationist approach but higher in applied and translational stages and research that follows an effect-centric confirmatory approach.

The Expected Replication Rate

Given its variability and ambiguity, it is unclear how to formally specify the optimal replication rate. Indeed, Tyner et al. (2026) concluded that "the optimal replicability rate is not known" (p. 148; see also Nosek, quoted in Fox, 2026). In other words, it is unclear what the observed replication rate should be. Instead, we can consider what people expect the replication rate to be. We can then use this expected replication rate as a benchmark against which to judge whether the observed replication rate is too low, too high, or consistent with expectations. For example, commenting on Tyner et al.'s (2026) work, Nosek (quoted in Jones, 2026) assumed that scientists would be surprised by a lower than expected replication rate:

We don't know what the optimal level of replicability is....The important part is not the number — it's that scientists say "wow, I had presumed that published findings are more repeatable than they are." (p. 38)

Indeed, several commentators have argued that the replication crisis occurred because observed replication rates are markedly lower than "expected or desired" (Nosek et al., 2022, p. 724; see also Munafò et al., 2017, p. 1; Open Science Collaboration, 2015, p. 7). Hence, as Tyner et al. (2026) explained, "the problem to solve is not unreplicability per se, it is overconfidence" (p. 148; see also Nosek, quoted in Fox, 2026).

Consistent with this view, there is evidence that researchers tend to overestimate replication rates. For example, Table 1 provides data from six studies that have used prediction markets and forecasting surveys to estimate expected replication rates and compare them with observed replication rates.

Table 1 shows that people tend to overestimate replication rates. Importantly, however, the extent of this overestimation does not appear to be particularly extreme. On average, people estimate the replication rate to be around 13 percentage points higher than the observed replication rate (i.e., ~63% rather than ~50%). It is debatable whether a 13-point overestimation represents sufficient grounds for a replication "crisis" (Rubin, 2023, p. 4).

It is also important to appreciate that the expected replication rate reflects people's beliefs, and that their beliefs may not represent the optimal replication rate for scientific purposes. Hence, a discrepancy between the observed and expected replication rate does not necessarily imply that we need to change our research practices. Indeed, if community expectations are substantially different from both observed and optimal replication rates, then they will be not only unrealistic but also suboptimal! Hence, we end up in a situation in which (a) the optimal replication rate is unclear and (b) the expected replication rate may be misleading.

Comment from a Co-Author

is one of the authors of the Tyner et al. (2026) paper. He recently shared his thoughts about Tyner et al.'s (2026) work on BlueSky (Haber, 2026, April 10). His comments reflect some of the issues discussed here:

Question now becomes "does the replication rate differ from what people *expect* it would be."

There's definitely a problem if those don't match, but unclear what the problem is.

There is no obviously correct benchmark for what that replication rate should be (and be skeptical of anyone who is sure they know what it is).

Heck, with all the time I've spent in this work, I am still truly not sure if/how much we should be bothered by the rates found.

Addressing the Replication Crisis

The typical response to the replication crisis has been to design and promote changes to research practices to try to increase observed replication rates. This response implies that the replication crisis is a methodological problem, and that methodological solutions are required to increase observed replication rates to a more optimal level. However, if the optimal replication rate is unknown, then the success or failure of this response will be unclear. Again, the problem here is one of a logically incomplete comparison (Rubin, 2023, p. 4).

An alternative perspective is to view the replication crisis as a problem of researchers' collective "overconfidence" (Tyner et al., 2026, p. 148) caused by their unrealistically high expected replication rates. This more social psychological perspective avoids the problem of an incomplete comparison because we are able to measure both observed and expected replication rates. It also implies an alternative solution to the replication crisis. Specifically, the replication crisis can be addressed by not only increasing the observed replication rate but also decreasing the expected replication rate. From this perspective, it may be helpful to educate scientists, the public, and the media that (a) there is no clear optimal replication rate and (b) a ~50% replication rate is realistic.

But What About False Positives?

Some may balk at the proposal to reduce people's expected replication rate because they view unexpectedly low replication rates as implying that original studies suffer from low quality methodology and questionable research practices that produced false positive results. However, it is worth referring to Tyner et al.'s (2026) sage advice when interpreting replication failures and successes:

Replication failures are not necessarily due to the original findings having low credibility. Low replication rates can also be due to false negatives, poorly designed replications, selecting only positive results for replication, and differences between original and replication studies that are initially perceived as unimportant. (p. 143)

A single failure to replicate does not justify concluding that the original research was wrong …. Even if the replication appeared to be testing the same research question, there could be differences in the methodology, sample or context that are unrecognized moderators of the outcome. In addition, even if the replication researchers were diligent in conducting the research, there could be unrecognized errors or flaws in implementing the replication protocol that interfered with observing the outcome. (p. 147)

A single successful replication does not justify concluding that the original research was correct …. The replicability of an effect is not the same as the validity of its interpretation. Original and replication studies may share confounds, faulty measures or other design weaknesses that produce replicable, but misinterpreted, outcomes. (p. 148)

Conclusion

In summary, it's difficult to argue that replications rates are suboptimal because it is unclear what the optimal replication rate should be. It's easier to argue that replication rates are lower than expected, but the evidence suggests that they are only around 13 percentage points below community expectations, and it's unclear whether this degree of discrepancy is problematic, especially given that community expectations may not be the best guide for scientific practice. Consequently, the rationale for a replication "crisis" remains debatable. Finally, if the "crisis" is conceived as a problem of collective "overconfidence" (Tyner et al., 2026, p. 148) rather than inadequate methodology, then it may be more effective to adjust community expectations to better reflect realistic replication rates. In other words, the replication crisis may serve as a valuable lesson in scientific humility.

References

Camerer, C. F., Dreber, A., Forsell, E., Ho, T. H., Huber, J., Johannesson, M., ... & Wu, H. (2016). Evaluating replicability of laboratory experiments in economics. Science, 351(6280), 1433–1436. https://doi.org/10.1126/science.aaf091

Camerer, C. F., Dreber, A., Holzmeister, F., Ho, T. H., Huber, J., Johannesson, M., ... & Wu, H. (2018). Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nature Human Behaviour, 2(9), 637-644. https://doi.org/10.1038/s41562-018-0399-z

Chandrashekar, S. P., Viganola, D., Dreber, A., Johannesson, M., Pfeiffer, T., Siegel, A., & Feldman, G. (2026). Using prediction markets and forecasting surveys to predict 28 replication outcomes of classic articles in social psychology and judgement and decision making. Royal Society Open Science, 13(1), 250377. https://doi.org/10.1098/rsos.250377

Devezer, B., & Buzbas, E. O. (2023). Rigorous exploration in a model-centric science via epistemic iteration. Journal of Applied Research in Memory and Cognition, 12(2), 189–194. https://doi.org/10.1037/mac0000121

Dreber, A., Pfeiffer, T., Almenberg, J., Isaksson, S., Wilson, B., Chen, Y., ... & Johannesson, M. (2015). Using prediction markets to estimate the reproducibility of scientific research. Proceedings of the National Academy of Sciences, 112(50), 15343–15347. https://doi.org/10.1073/pnas.1516179112

Feest, U. (2024). What is the replication crisis a crisis of? Philosophy of Science, 91(5), 1361-1371. https://doi.org/10.1017/psa.2024.2

Fox, J. (2026, April 11). When being right less than half the time…is fine. Bloomberg. https://www.bloomberg.com/opinion/articles/2026-04-11/when-being-right-less-than-half-the-time-is-fine

Gordon, M., Viganola, D., Dreber, A., Johannesson, M., & Pfeiffer, T. (2021). Predicting replicability—Analysis of survey and prediction market data from large-scale forecasting projects. PLOS One, 16(4), e0248780. https://doi.org/10.1371/journal.pone.0248780

Jones, N. (2026, April 2). Why science has a credibility problem — and how to address it. Nature, 652, 36. https://doi.org/10.1038/d41586-026-00972-4

Lakatos, I. (1978). The methodology of scientific research programmes (Philosophical Papers, Volume I). Cambridge University Press.

Mayrhofer, R., Büchner, I. C., & Hevesi, J. (2024). The quantitative paradigm and the nature of the human mind. The replication crisis as an epistemological crisis of quantitative psychology in view of the ontic nature of the psyche. Frontiers in Psychology, 15:1390233. https://doi.org/10.3389/fpsyg.2024.1390233

Mody, F., Wilkinson, D. P., Fraser, H., Fidler, F., Bishop, M. M., Burgman, M., … Pfeiffer, T. (2026). Large-scale human predictions of the replicability of published social and behavioural science papers – a multi- study analysis. MetaArXiv. https://doi.org/10.31222/osf.io/vgyed_v1

Munafò, M. R., Nosek, B. A., Bishop, D. V., Button, K. S., Chambers, C. D., Percie du Sert, N., ... & Ioannidis, J. (2017). A manifesto for reproducible science. Nature Human Behaviour, 1(1), 1-9. https://doi.org/10.1038/s41562-016-0021

Nak, J., van Dongen, N. N. N., Woodward, J. F., Scheel, A. M., Haig, B., van Bork, R., Blanken, T. F., de Ron, J., Finnemann, A., Haslbeck, J., Francken, J., Eronen, M., van Lissa, C. J., Dijstelbloem, H., & Borsboom, D. (2026). Establishing psychological phenomena: An alternative methodological perspective on scientific practice. PsyArXiv. https://doi.org/10.31234/osf.io/rh9cu_v1

Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., ... & Vazire, S. (2022). Replicability, robustness, and reproducibility in psychological science. Annual Review of Psychology, 73, 719–748. https://doi.org/10.1146/annurev-psych-020821-114157

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716. https://doi.org/10.1126/science.aac4716

Popper, K. R. (1962). Conjectures and refutations: The growth of scientific knowledge. Routledge.

Popper, K. R. (1966). Some comments on truth and the growth of knowledge. Studies in Logic and the Foundations of Mathematics, 44, 285-292. https://doi.org/10.1016/S0049-237X(09)70596-8

Rubin, M. (2023). Questionable metascience practices. Journal of Trial and Error, 4(1), 5–20. https://doi.org/10.36850/mr4

Rubin, M. (2025). The replication crisis is less of a "crisis" in Lakatos' philosophy of science than it is in Popper's. European Journal for Philosophy of Science, 15(5). https://doi.org/10.1007/s13194-024-00629-x

Tyner, A. H., Abatayo, A. L., Daley, M., et al. (2026). Investigating the replicability of the social and behavioural sciences. Nature, 652, 143–150. https://doi.org/10.1038/s41586-025-10078-y

Subscribe now

Share