Citation counts are similar to social media “likes” — they are always positive. But what if we had negative citations?

Blair Fix

Scientists live and die by their scientific 'impact'. For the
uninitiated, 'impact' is a measure of a scientist's contribution to
their field. While there are many measures of scientific impact, almost
all of them focus (in some way) on *citations*. So if more people cite
your papers, you have more scientific 'impact'.

The idea behind counting citations is that they quantify other people's
engagement with your ideas. It's similar to counting 'likes' on social
media. And as with social media, there is (currently) no such thing as a
scientific 'dislike'. There is no such thing as a *negative* citation.

But what if there was? What if citations could be both positive and
negative? This, I think, would throw a much-needed wrench in how we
measure impact. It would be uncomfortable for researchers, but probably
good for science.

### Not all press is good press

The idea behind negative citations is simple --- we need to distinguish
between *praise* and *criticism*. If I cite a paper while praising it,
this should count as a *positive* citation. But if I cite a paper while
criticizing it, this should count as a *negative* citation.

Similarly, if I test a theory and find empirical support, my citation to
the theory paper should be *positive*. But if I test a theory and find
*no* empirical support, my citation to the theory paper should be
*negative*.

The use of both positive and negative citations is, I think, more in
line with how science is done. We don't judge theories solely by the
*number* of times they've been tested. Instead, we judge theories by the
cumulative weight of evidence, both good and bad. So a theory that is
well tested --- yet uniformly found to be wrong --- does not count as a
'good' theory. It is a theory that should be abandoned.

What if we judged the scientific impact of *individuals* no differently
than how we judge a scientific theory? We would add positive citations
and subtract negative citations, just as we weight evidence for and
against a theory.

So when I write a [paper that is critical of human capital
theory](http://www.paecon.net/PAEReview/issue86/Fix86.pdf){target="_blank"
rel="noopener"}, my references to human capital papers should count
*negatively* to their citation count. Similarly, when someone [tests
revealed preference theory and finds no
support](https://www.jstor.org/stable/2957744?seq=1#page_scan_tab_contents){target="_blank"
rel="noopener"}, their references to revealed preference theorists
should count negatively.

Allowing citations to be both positive and negative would completely
change how we measure scientific impact. But would it be for the better?
I have mixed feelings.

### Problems with negative citations

You've likely thought of some problems with negative citations. An
obvious problem is that we need algorithms that can distinguish between
a positive and negative citation. This would require natural language
processing and the ability to parse context. Understanding the
positive/negative context of a citation is certainly a difficult task.
But given the success of natural language parsers like
[Watson](https://en.wikipedia.org/wiki/Watson_(computer)){target="_blank"
rel="noopener"}, this type of algorithm isn't a pipe dream.

There's also the problem of how we decide what counts as a positive or
negative citation. I recognize that there's a subjective element to this
decision. And many citations may be neither positive nor negative.
They're what I call 'citation dumps' --- an obligatory dump of
information, referenced without comment. Maybe citation dumps should
count neither positively or negatively to citation counts?

The more I think about actually implementing a positive/negative
citation scheme, the more difficult it seems. But still, it would be
interesting to at least *try* to distinguish between positive and
negative citations.

The biggest problem with negative citations, however, is not creating
the algorithm. Instead, it's a much deeper problem that plagues all of
science. We call it the *tyranny of the majority*.

Suppose a majority of researchers all believe in a certain theory (that
is wrong). The person who advances an alternative (correct) theory will
initially face a torrent of negative citations.

I don't know a way to fix this problem. It's just a fact that all new
ideas start with few proponents, and must battle the inertia of the
establishment. Granted, most new ideas are wrong. But some are right.

When science works as it should, correct ideas win the day. But the
battle over new ideas can take a long time. History is littered with
correct ideas whose progenitors died before the idea gained acceptance.
The classic example is [Alfred
Wegener](https://en.wikipedia.org/wiki/Alfred_Wegener){target="_blank"
rel="noopener"}, who first proposed the idea of continental drift.
Wegener's ideas were ridiculed during his lifetime. But 30 years after
Wegener's death, the consensus changed. Continental drift is now an
established fact.

Although I like the idea of negative citations, I worry that it would
exacerbate the tyranny of the majority. Would the spectre of a negative
citation make young researchers less likely to challenge authority?
Would negative citations foster groupthink? I honestly don't know.

While I worry about these problems, I think negative citations would
allow fascinating analysis of scientific progress. Take [Thomas
Kuhn's](https://en.wikipedia.org/wiki/Thomas_Kuhn){target="_blank"
rel="noopener"} idea of a [paradigm
shift](https://en.wikipedia.org/wiki/Paradigm_shift){target="_blank"
rel="noopener"}. This happens when an accepted set of ideas (like
Newtonian physics) is replaced by a new set of ideas (like quantum
physics). I think it would be fascinating to measure paradigm shifts
using negative citations.

### Measuring paradigm shifts with positive and negative citations

Let's imagine how using negative citations might quantify a paradigm
shift.

We start with an accepted paradigm that has many citations, most of them
positive. Then something happens to trigger a paradigm shift. Scientists
grow more critical of the old paradigm, and more negative citations crop
up. Eventually, almost all references to the old paradigm are negative.

Measured using positive and negative citations, the death of an old
scientific paradigm might look like this:

![old
paradigm](https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2019/06/old_paradigm.png?resize=723%2C578&ssl=1){.alignnone
.size-full .wp-image-478 loading="lazy" decoding="async"
attachment-id="478"
permalink="https://economicsfromthetopdown.com/old_paradigm/"
orig-file="https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2019/06/old_paradigm.png?fit=2400%2C1920&ssl=1"
orig-size="2400,1920" comments-opened="1"
image-meta="{\"aperture\":\"0\",\"credit\":\"\",\"camera\":\"\",\"caption\":\"\",\"created_timestamp\":\"0\",\"copyright\":\"\",\"focal_length\":\"0\",\"iso\":\"0\",\"shutter_speed\":\"0\",\"title\":\"\",\"orientation\":\"0\"}"
image-title="old_paradigm" image-description=""
image-caption="old paradigm
"
medium-file="https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2019/06/old_paradigm.png?fit=300%2C240&ssl=1"
large-file="https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2019/06/old_paradigm.png?fit=723%2C578&ssl=1"
width="723" height="578" recalc-dims="1"}

At the same time that the old paradigm dies, a new one is born. This new
theory starts off with no citations. Then, as the old guard learns of
it, the new paradigm gets a torrent of criticism. So its citation count
goes negative. But over time, the new paradigm gains acceptance and the
citation count turns positive.

Measured using positive and negative citations,the birth of a new
paradigm might look like this:

![new
paradigm](https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2019/06/new_paradigm.png?resize=723%2C578&ssl=1){.alignnone
.size-full .wp-image-477 loading="lazy" decoding="async"
attachment-id="477"
permalink="https://economicsfromthetopdown.com/new_paradigm/"
orig-file="https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2019/06/new_paradigm.png?fit=2400%2C1920&ssl=1"
orig-size="2400,1920" comments-opened="1"
image-meta="{\"aperture\":\"0\",\"credit\":\"\",\"camera\":\"\",\"caption\":\"\",\"created_timestamp\":\"0\",\"copyright\":\"\",\"focal_length\":\"0\",\"iso\":\"0\",\"shutter_speed\":\"0\",\"title\":\"\",\"orientation\":\"0\"}"
image-title="new_paradigm" image-description=""
image-caption="new paradigm
"
medium-file="https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2019/06/new_paradigm.png?fit=300%2C240&ssl=1"
large-file="https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2019/06/new_paradigm.png?fit=723%2C578&ssl=1"
width="723" height="578" recalc-dims="1"}

It would be fascinating to apply this accounting to real theories. It
makes me wonder what we'd find.

### Negative citations are fine for theories ... but what about people? {#negative-citations-are-fine-for-theories-but-what-about-people}

I have mixed feelings about applying negative citations to people. On
the one hand, I would be ecstatic if the citation count of human capital
theorists (I won't name names) went deep into the red. But I wouldn't
want to be on the receiving end of a torrent of negative citations.

But while negative citations may be uncomfortable *scientists*, my gut
feeling is that they would be good for *science*.

I'd like to hear your thoughts. Are negative citations a good idea? Or
am I off my rocker?

------------------------------------------------------------------------

#### Support this blog

Economics from the Top Down is where I share my ideas for how to create
a better economics. If you liked this post, please consider becoming a
patron. You'll help me continue my research, and continue to share it
with readers like you.

[![patron_button](https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2023/05/supporter_button-1.png?w=220&ssl=1){.size-thumbnail
.wp-image-1180 .aligncenter decoding="async" attachment-id="1180"
permalink="https://economicsfromthetopdown.com/patron_button/"
orig-file="https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2020/02/patron_button.png?fit=434%2C102&ssl=1"
orig-size="434,102" comments-opened="1"
image-meta="{\"aperture\":\"0\",\"credit\":\"\",\"camera\":\"\",\"caption\":\"\",\"created_timestamp\":\"0\",\"copyright\":\"\",\"focal_length\":\"0\",\"iso\":\"0\",\"shutter_speed\":\"0\",\"title\":\"\",\"orientation\":\"0\"}"
image-title="patron_button" image-description="" image-caption=""
medium-file="https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2020/02/patron_button.png?fit=300%2C71&ssl=1"
large-file="https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2020/02/patron_button.png?fit=434%2C102&ssl=1"
recalc-dims="1"}](https://economicsfromthetopdown.com/membership/){target="_blank"
rel="noopener"}

------------------------------------------------------------------------

#### Stay updated

Sign up to get email updates from this blog.

:::: {.jetpack_subscription_widget}
## {#section .widgettitle}

::: {.wp-block-jetpack-subscriptions__container}
Email Address

Keep me up to date
:::
::::

------------------------------------------------------------------------

[![](https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2021/03/by.png?w=150&ssl=1){.aligncenter
decoding="async"
recalc-dims="1"}](http://creativecommons.org/licenses/by/4.0/){rel="license"} 
This work is licensed under a [Creative Commons Attribution 4.0
License](http://creativecommons.org/licenses/by/4.0/){rel="license"}.
You can use/share it anyway you want, provided you attribute it to me
(Blair Fix) and link to [Economics from the Top
Down](https://economicsfromthetopdown.com/).

------------------------------------------------------------------------

Scientists live and die by their scientific ‘impact’. For the uninitiated, ‘impact’ is a measure of a scientist’s contribution to their field. While there are many measures of scientific impact, almost all of them focus (in some way) on

 citations

. So if more people cite your papers, you have more scientific ‘impact’. The idea behind counting citations is that they quantify other people’s engagement with your ideas.

Messaggi di Rogue Scholar

What If Scientific Impact Could Be Negative?