A do-or-die moment for the scientific enterprise

Today, our article "The entities enabling scientific fraud at scale are large, resilient, and growing rapidly" was published in PNAS. The article will only be fully open-access in six months (damn you, publishing fees), but I've made the PDF available here (you can download the supplementary materials on the PNAS page).

I'll use this blog post to summarize and opine (what I write here reflects my own thinking about this study and not necessarily that of my co-authors or employer). In our article, we demonstrate through a series of vignettes that…

1. Editors abuse their positions of authority to collude with authors to publish problematic articles en masse. Using data from journals where the published article is labeled with the handling editor (i.e., PLOS One and Hindawi journals), we show that certain editors handle problematic articles far more often than can be explained by chance. We also show that certain authors seem to have a preference for having their articles handled by these flagged editors. For PLOS One, we identify a network in which these flagged editors were all handling each other's submissions to the journal.

1. Networks of image duplication can be thousands of articles wide and these articles tend to appear in the same journals at around the same time. We reconstruct a network of image duplication using PubPeer comments made before January 2024. We show that the articles connected by shared images also tend to get published at around the same time and in the same venue. This suggests that paper mills are capable of both producing articles and getting them published in a highly coordinated fashion–wholesale, not custom-made. Publishers often claim that paper mill products are published because they have slipped through the cracks. This vignette instead suggests a model where paper mills have relatively open pathways into journals–likely facilitated through the knowing cooperation of editors, as suggested in the first vignette.

1. Broker organizations are capable of placing articles in journals on demand and adapt well under adversarial conditions. The Academic Research Development Association (ARDA) is an Indian business that offers "Conferences and Meetings", "Journal Publications" and "Thesis/Article Writing". ARDA has been described previously by us, Rudi de Lange and David Bimler. ARDA has, over the last eight years, maintained a list of journals on its website in which it can guarantee publication of clients' articles for a fee. These journals (listed in Table S5) run the gamut of indexing status and publisher reputability. We show that this list has evolved and grown over time, often apparently in response to journals being de-indexed (i.e., as soon as one of ARDA's journals was de-indexed, it was quickly replaced by another journal). We download the archives of five journals on ARDA's menu of offerings (Table S6) and show that these journals have collectively published more than 20,000 articles from around the world, most of which are well outside of the stated scope of the journal (Table S7, see Footnote 1 below).

1. Some fields appear to be more vulnerable to paper mill activity than others. While paper mill activity is well-documented in some fields (e.g., cancer biology), other fields have seen fewer reports of industrialized systematic fraud. We wondered whether this trend was attributable to baseline differences in the incidence of industrialized scientific fraud across fields or to biases in our collective detection of paper mill products. To get closer to an answer, we look at a small sample of closely-related fields in RNA biology research. We found that the rates at which corrections and errata were published were similar across these fields, indicating that the extant literature was revisited at similar rates. However, retraction rates varied wildly. Certainly, systematic scientific fraud can find purchase in just about any field and the search for paper mill products is definitely better-developed in some disciplines. However, our findings suggest that some fields are indeed more vulnerable to widespread contamination of their literature than others. We surmise that paper mills have a preference for fields in which plausible manuscripts can be generated rapidly by mixing and matching novel elements into formulaic templates. Suchak et al. and Stender et al. have recently described exactly this pattern in epidemiological studies.

1. Publishers understand that systematic fraud underlies the bulk of their integrity issues. We show that most retractions are now issued in batches, alongside ten or more retractions in the same journal on the same day. If you've spent any amount of time reporting bunches of related problematic articles to publishers, you've probably had an editor or research integrity specialist explain to you that they will only consider articles individually on their own merits when making decisions on editorial action. This data reveals that when publishers do issue retractions, they largely retract many articles at once for related reasons. Whereas the rarity of retractions (see next vignette) paints a picture of publication misconduct occurring in isolated events, publishers see increasingly frequent large-scale breaches of publication integrity.

1. The integrity measures used to contain systematic scientific fraud are dwarfed in scale by the problem itself. We assemble a corpus of suspected paper mill products from dozens of smaller corpora of articles experts have labeled as likely paper mill products. Bearing in mind that there are definite biases across fields and time in our assembled corpus (and that there is still no standard definition of what a "paper mill" actually is), we find that this corpus has been growing in size (annual count doubling every 1.5 years) at a rate far eclipsing the growth rates of all scientific articles (doubling every 15 years), as well as PubPeer-commented articles and retracted articles (each doubling every 3-4 years). Moreover, suspected paper mill products in this corpus already outnumber annual retractions, without even taking into account that this corpus represents just a small fraction of what is probably out there. Based on current trends, we estimate that only around one in four paper mill products will ever be retracted. The situation with de-indexing is similarly grim: while literature aggregators like MEDLINE, Scopus and Web of Science (WoS) routinely trumpet their commitment to maintaining the integrity of their products, we show that they de-index fewer than 100 journals every year. For context, there are more than 22,000 journals in WoS' 2025 Journal Citation Reports, 600 more than the year before. We estimate that fewer than 10% of all paper mill products will ever reside in a de-indexed journal.

There is far more that we don't understand about the industry of scientific fraud than what we do. In particular, inferring the exact magnitude and disciplinary scope of the problem is a nigh-impossible task. I hope that, if anything, we've shown in our article that industrialized scientific fraud can and should be studied systematically (see perspectives by Byrne et al. and Abalkina et al.).

We start our article by conceptualizing the scientific enterprise as one large public goods game. An idealized public goods game might proceed like this: you have five players, each of which has $10. At the start of the turn, each player puts their $10 into a public "pot", which then has $50 in it. The pot then multiplies in value by some factor and the pot is redistributed equally to all players–let's say 1.5 times, such that $50 becomes $75 and each player gets back $15. In this turn, each player has then made a profit of $5 (a $10 contribution for a $15 return). This game would not be very interesting if not for a voluntary contributions mechanism: each player gets to decide how much they contribute to the pot. 

Now imagine a turn where four players contribute $10 to the pot and one player contributes nothing. The $40 pot becomes $60, which is then redistributed among all five players, each receiving $12. The four players that initially contributed $10 made a profit of $2, but the player that initially contributed nothing made a profit of $12. The act of choosing to contribute less than other players despite having the means to contribute in full is called "defection". In this scenario, the total profit among all players is less than before, but the defecting player actually sees a greater profit than before.

In repeating laboratory public goods games, players inevitably realize that defection is the winning strategy and begin to contribute less to the pot. There is usually a core of "cooperating" players that continue to make as large a contribution as they can to the pot, but most player contributions decay over time, the total payout from the pot gets smaller and the number of defectors increases as players realize that they make less profit by fully cooperating.

The scientific enterprise is obviously far larger and more complex than these idealized public goods games, but the process at work is similar: everyone makes some contribution to the pot and receives some reward realized through the collective sum of contributions (see Footnote 2 below). Because science stimulates economies, enhances human well-being and tickles our intellectual desires, we all see some payout from the pot in an abstract sense. However, the benefits to participating in the great scientific public goods game can be much more concrete. Scientists can earn long, rewarding careers. Private-sector firms can capitalize on new technologies. States collect more taxes from a healthier, wealthier workforce. Of course, there is also a lot of money to be made in the business of disseminating scientific literature.

Here's what I've come to understand over the last three years that we've been conducting and refining our study and fighting to get it published: the scientific enterprise is now witness to widespread, organized defection from the scientific public goods game. Large swaths of players, among them many scientists, reviewers, editors and publishers, are choosing to no longer make genuine contributions to the pot. Parascientific organizations (like ARDA, paper mills and the groups of collaborating editors) now facilitate and profit from mass-scale defection. Many scientists, especially in countries where the resources for doing genuine science are more scarce, are now trained in contexts where defection is the normative behavior. 

Some model public goods games integrate a mechanism by which defectors are punished. While this can be effective at mitigating defection under certain circumstances, our study shows that the punitive measures employed to enforce science integrity, like retractions and de-indexing, are currently applied far too infrequently to meaningfully increase the costs of defection. As ARDA shows, these measures can also be easily sidestepped.

Meanwhile, the United States government is dismantling research support infrastructure and funding wholesale, defecting from their role in our immense public goods game and ensuring that contributions by taxpayers will also wither. In effect, competition for resources will only grow more fierce and it will become more and more difficult to make genuine contributions. Those unwilling to defect may likely leave the game or be forced out. 

If the model public goods game offers any prognostication, it's that the current paradigm, where defection is the winning strategy, ensures that genuine contributions will only decay from here. We will all be worse off for it. Anyone that has studied industrialized scientific fraud has seen the future of our scientific enterprise, as has anyone that has had their funding rescinded by the United States government, as has anyone that has been told that they must publish X papers annually, as has anyone that has ever bought a paper mill product. 

I have seen the future of science. It is ruled by bitter competition instead of collaboration, pageantry instead of exploration. Bright minds beginning careers in science will be taught to debase their training for drudgerous pursuit of meaningless metrics. Those willing to toil over genuine questions will necessarily lose out to those that can furnish cheap answers.

Many scientists worldwide already inhabit this reality. So, we all may. If this vision comes to pass, humanity will lose its most potent engine for progress and its most abundant source of wonder.

I have few words left other than every scientist's most familiar motto: we must act now, before it is too late.

Footnote 1:

The complete archives of the five journals we downloaded are available here:

The Journal of Pharmaceutical Negative Results 
https://doi.org/10.5281/zenodo.15272024
PalArch's Journal of Archaeology of Egypt/Egyptology
https://doi.org/10.5281/zenodo.16688437
Res Militaris
https://doi.org/10.5281/zenodo.16672845
Russian Law Journal
https://doi.org/10.5281/zenodo.15276443
HIV Nursing
https://doi.org/10.5281/zenodo.15278348

Footnote 2:

It is difficult to encourage genuine contributions and disincentivize defection without knowing what everyone's contributions actually are. Faced with the intractable complexity of the scientific enterprise, infometricians have developed a variety of methods by which contributions to the scientific public goods game can be measured. These include publication and citation counts, the h-index, journal impact factor and institutional rankings. The trouble with these measures is that they don't actually measure the contributions a player has made, but rather what they have to show for it. You can't easily count the hours someone has spent in the lab or quantify their brilliance or predict their propensity for training more brilliant scientists, but you can count the number of articles they have published and how often they have been cited. 

In effect, these metrics measure the revenue someone reaps from the pot, not their contributions to it. While you can put in a lot of work to produce a handful of articles, you can produce many more articles by being a more senior, supervisory scientist with a large team of scientists working under you. Having a large team may give some indication that you have made some outstanding contributions in your time, but it is a far more direct indicator of the revenue you have collected from the pot.

Defection can inflate this revenue drastically, which in turn inflates these indicators. A lack of construct validity is problematic by itself, but these metrics are also used to determine how the pot is doled out–who gets hired, who gets tenure, who gets funding.

Footnote 3:

The header image for this post is the painting Italian Cemetery at Dusk by Albert Chmielowski (1880).