Here are some tricks for scraping the web with R and Python.

Blair Fix

This week I’m sharing some tips for web scraping. The internet is overflowing with information. But it usually takes some work to get the data into a usable form. Here are some tricks for making that happen. Scraping starts with downloading   Every scrape job starts with a download. When you view a website on your computer, your browser is just displaying data that it downloaded from the website’s server.

Tips for  Web Scraping in R and Python

Does hierarchy mediate the growth of top incomes?

Download PDF Abstract   Although the determinants of income are complex, the results are surprisingly uniform. To a first approximation, top incomes follow a power-law distribution, and the redistribution of income corresponds to a change in the power-law exponent. Given the messiness of the struggle for resources, why is the outcome so simple?

Redistributing Income Through Hierarchy

An interview on the ‘Platform’ podcast.

Last week I had the pleasure to chat with Rachel Donald on her podcast
['Platform'](https://www.platformenterprise.com/){target="_blank"
rel="noopener"}. The show features people who are 'pissed off with
capitalism'. Its goal is to give a platform for under-reported topics
that deserve traction and attention.

<figure class="wp-block-image size-large">
<a
href="https://www.platformenterprise.com/p/grassroots-economics-for-a-better"
target="_blank" rel="noopener"><img
src="https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2021/10/podcast_image.png?w=723&amp;ssl=1"
decoding="async" data-recalc-dims="1" /></a>
</figure>

Rachel and I talked about a wide range of topics, but I particularly
enjoyed discussing my research on hierarchy and income. Most people take
it for granted that when they go to work, they'll take orders from a
boss who earns more than them. And they may give orders to a subordinate
who earns less. That's just how companies work ... a fact of life.

Rachel was blown away that mainstream economists don't engage with these
facts of hierarchy. Or if they do, they dismiss them as an outcome of
'human capital'. (Perhaps you didn't know, but your boss is more
'productive' than you.)

On another note, I've been listening to the Platform back catalogue, and
am enjoying the material. Here are some recent highlights:

- [Carey King: The Myth of
 Efficiency](https://www.platformenterprise.com/p/the-myth-of-efficiency){target="_blank"
 rel="noopener"}
- [Steve Keen: The Economics of Climate
 Change](https://www.platformenterprise.com/p/the-economics-of-climate-change){target="_blank"
 rel="noopener"}
- [James Dyke: Net-Zero Won't Save
 Us](https://www.platformenterprise.com/p/net-zero-wont-save-us){target="_blank"
 rel="noopener"}

If you like Rachel's work, consider supporting her on
[Substack](https://www.platformenterprise.com/){target="_blank"
rel="noopener"}. (I just did.) Like me, Rachel is determined not to
paywall her content. So she relies on goodwill to support the show.

------------------------------------------------------------------------

#### Support this blog

Economics from the Top Down is where I share my ideas for how to create
a better economics. If you liked this post, consider becoming a patron.
You'll help me continue my research, and continue to share it with
readers like you.

[![patron_button](https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2023/05/supporter_button-1.png?w=220&ssl=1){.aligncenter
decoding="async"
recalc-dims="1"}](https://economicsfromthetopdown.com/membership/){target="_blank"
rel="noopener"}

------------------------------------------------------------------------

#### Stay updated

Sign up to get email updates from this blog.

:::: {.jetpack_subscription_widget}
## {#section .widgettitle}

::: {.wp-block-jetpack-subscriptions__container}
Email Address

Keep me up to date
:::
::::

------------------------------------------------------------------------

[![](https://economicsfromthetopdown.files.wordpress.com/2021/03/by.png?w=150){.aligncenter
decoding="async"
recalc-dims="1"}](http://creativecommons.org/licenses/by/4.0/){rel="license"} 
This work is licensed under a [Creative Commons Attribution 4.0
License](http://creativecommons.org/licenses/by/4.0/){rel="license"}.
You can use/share it anyway you want, provided you attribute it to me
(Blair Fix) and link to [Economics from the Top
Down](https://economicsfromthetopdown.com/).

------------------------------------------------------------------------

Last week I had the pleasure to chat with Rachel Donald on her podcast ‘Platform’. The show features people who are ‘pissed off with capitalism’. Its goal is to give a platform for under-reported topics that deserve traction and attention. Rachel and I talked about a wide range of topics, but I particularly enjoyed discussing my research on hierarchy and income.

Talking Hierarchy With Rachel Donald

Chris Mouré has a new paper on big-tech battles. Here are the highlights.

Political economist Chris Mouré has a new paper out in the *Review of
Capital as Power*. It's called
['Soft-wars'](https://capitalaspower.com/2021/10/moure-soft-wars-a-capital-as-power-analysis-of-googles-differential-power-trajectory/){target="_blank"
rel="noopener"}, and it is a fascinating case study of the behavior of
big tech.

The story starts in 2011, when Microsoft led a \$4.5 billion consortium
purchase of Nortel and Novel. Later than year, Google responded by
buying Motorola for \$12.9 billion. The funny thing is that Google then
proceeded to *sell off* what it had just bought. By 2014, almost nothing
was left of Google-owned Motorola. Nothing except *patents*. And that,
Mouré thinks, was the whole point.

Mouré argues that this acquisition war was ultimately a battle over
intellectual property. Google and Microsoft were competing to control
the mobile market. And the way to do that was not to 'produce' anything.
It was to command property rights.

The timing of the 2011 patent war, Mouré notes, was no coincidence. It
corresponded with the moment when Google's profits caught up to
Microsoft. Figure [1](#fig-google-dif) shows the trend. This is Mouré's
analysis of 'differential profit' --- the profit of Microsoft and Google
measured relative to the average profit of the S&P 500. You can see that
Google entered the 21st century as a bit player. But by the 2010s,
Google was a behemoth whose profits matched those of Microsoft.

[]{#fig-google-dif}

<figure class="wp-block-image size-large">
<img
src="https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2021/10/softwarsfigure3.png?w=723&amp;ssl=1"
decoding="async" data-recalc-dims="1" />
<figcaption>Figure 1: Google catches Microsoft. Chris
Mouré’s analysis of differential profit — profit relative to the S&amp;P
500 average. Source: <a
href="https://capitalaspower.com/2021/10/moure-soft-wars-a-capital-as-power-analysis-of-googles-differential-power-trajectory/"
target="_blank" rel="noopener">Soft-wars</a>.</figcaption>
</figure>

Looking closer at the data, Mouré notices an interesting shift in
behavior that happened after 2010. In the decade prior, Google's profit
was positively correlated with Microsoft's. But after 2010, the
correlation turned negative. In other words, when Google did well,
Microsoft suffered (and vice versa).

[]{#fig-cor-neg}

<figure class="wp-block-image size-large">
<img
src="https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2021/10/softwarsfigure4.png?w=723&amp;ssl=1"
decoding="async" data-recalc-dims="1" />
<figcaption>Figure 2: An antagonistic relation between Google
and Microsoft. This figure shows Chris Mouré’s analysis of
differential profit growth rates for Google and Microsoft. Prior to
2010, the growth rates were positively correlated. But after 2010, the
relation turned negative. Source: <a
href="https://capitalaspower.com/2021/10/moure-soft-wars-a-capital-as-power-analysis-of-googles-differential-power-trajectory/"
target="_blank" rel="noopener">Soft-wars</a>.</figcaption>
</figure>

What's going on here, Mouré argues, is that by 2010, Google and
Microsoft had become locked in a heated battle. By then, both firms
dominated their core businesses. Microsoft was a monopolist in the OS
business. And Google was a monopolist in the online ad business. And so
both companies were expanding into new territory. (In this case, mobile
computing.) Unsurprisingly, that let to mutual conflict. And so Google's
win became Microsoft's loss (and vice versa).

To a large extent, this competition played out in a war for
acquisitions. Google and Microsoft both spent billions buying up
competitors. What's impressive, Mouré notes, is that this buying spree
was highly coordinated. When Google bought big, so did Microsoft. Figure
[3](#fig-aquisition) tells the story.

[]{#fig-aquisition}

<figure class="wp-block-image size-large">
<img
src="https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2021/10/softwarsfigure8.png?w=723&amp;ssl=1"
decoding="async" data-recalc-dims="1" />
<figcaption>Figure 3: A coordinated battle for
aquisitions. This figure shows Chris Mouré’s analysis of
acquisitions spending by Microsoft and Google. Source: <a
href="https://capitalaspower.com/2021/10/moure-soft-wars-a-capital-as-power-analysis-of-googles-differential-power-trajectory/"
target="_blank" rel="noopener">Soft-wars</a>.</figcaption>
</figure>

Fast forward to 2018, which saw Microsoft buy LinkedIn for \$26 billion.
As with the Nortel buyout of 2011, Mouré thinks the timing of the
LinkedIn acquisition was no coincidence. It happened when Facebook
joined the monopoly game.

As Figure [4](#fig-facebook) shows, Facebook started the 2010s as a fly
on the 'Goog-soft' elephant. But by 2018, Facebook's profits had caught
up to both Microsoft and Google. Faced with this new adversary,
Microsoft repeated the same strategy: buy your competitor's main rivals.
And so Microsoft bought LinkedIn.

[]{#fig-facebook}

<figure class="wp-block-image size-large">
<img
src="https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2021/10/softwarsfigure9.png?w=723&amp;ssl=1"
decoding="async" data-recalc-dims="1" />
<figcaption>Figure 4: Facebook enters the monopoly
game. Chris Mouré’s analysis of differential profit — profit
relative to the S&amp;P 500 average. Source: <a
href="https://capitalaspower.com/2021/10/moure-soft-wars-a-capital-as-power-analysis-of-googles-differential-power-trajectory/"
target="_blank" rel="noopener">Soft-wars</a>.</figcaption>
</figure>

Mouré's analysis nicely illustrates a truism in economics: big
corporations behave nothing like economics textbooks say they should.
The textbooks say that profits should stem from productivity. But the
reality is that what big companies care about most is *restriction*. The
battle for profits is a battle over property rights --- the legal right
to restrict.

We all know that big tech is engaged in nefarious activity. The point is
that this the *norm*, not the exception. Econ textbooks say that firms
should play nice. In the real world, the big boys make their own rules.

### Further Reading {#further-reading target="_blank"}

Mouré, Chris. 2021. ['Soft-wars: A Capital-as-Power Analysis of Google's
Differential Power
Trajectory'](https://capitalaspower.com/2021/10/moure-soft-wars-a-capital-as-power-analysis-of-googles-differential-power-trajectory/){target="_blank"
rel="noopener"}. *Review of Capital as Power*, Vol. 2, No. 1,
pp. 71--90.

------------------------------------------------------------------------

#### Support this blog

Economics from the Top Down is where I share my ideas for how to create
a better economics. If you liked this post, consider becoming a patron.
You'll help me continue my research, and continue to share it with
readers like you.

[![patron_button](https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2023/05/supporter_button-1.png?w=220&ssl=1){.aligncenter
decoding="async"
recalc-dims="1"}](https://economicsfromthetopdown.com/membership/){target="_blank"
rel="noopener"}

------------------------------------------------------------------------

#### Stay updated

Sign up to get email updates from this blog.

:::: {.jetpack_subscription_widget}
## {#section .widgettitle}

::: {.wp-block-jetpack-subscriptions__container}
Email Address

Keep me up to date
:::
::::

------------------------------------------------------------------------

[![](https://economicsfromthetopdown.files.wordpress.com/2021/03/by.png?w=150){.aligncenter
decoding="async"
recalc-dims="1"}](http://creativecommons.org/licenses/by/4.0/){rel="license"} 
This work is licensed under a [Creative Commons Attribution 4.0
License](http://creativecommons.org/licenses/by/4.0/){rel="license"}.
You can use/share it anyway you want, provided you attribute it to me
(Blair Fix) and link to [Economics from the Top
Down](https://economicsfromthetopdown.com/).

------------------------------------------------------------------------

Political economist Chris Mouré has a new paper out in the

 Review of Capital as Power

. It’s called ‘Soft-wars’, and it is a fascinating case study of the behavior of big tech. The story starts in 2011, when Microsoft led a $4.5 billion consortium purchase of Nortel and Novel. Later than year, Google responded by buying Motorola for $12.9 billion.

Soft-wars

Does the deep past live with us, encoded in the structure of our institutions?

One of the challenges of studying the past is that time has a maddening way of erasing evidence. Take the origin of life — the ultimate cold case. Had the first organisms been wise enough to fossilize themselves, we would know exactly how life began. Fortunately, the first organisms

 did

preserve themselves, at least in rough form. That’s because life is itself a kind of time capsule.

Institution Size as a Window into Cultural Evolution

Yes you should read ‘Scale’. No, don’t believe everything it says.

![](https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2021/10/west_cover_2.jpg?w=200&ssl=1){decoding="async"
style="float:right; padding-left: 10px;" recalc-dims="1"}

For several years, people have been urging me to read Geoffrey West's
book *Scale*, claiming that his work overlaps with my own. Having read
the book, I largely agree (with some caveats below).

Geoffrey West is a physicist who's become famous for studying 'scaling'
effects across life on Earth. What is 'scaling'? It's best explained
through an example. Figure [1](#fig-kleiber) (taken from West's book)
shows how animals' metabolism varies with body mass. Given the tight
relation, scientists say that metabolic rate 'scales' with body mass.

[]{#fig-kleiber}

<figure class="wp-block-image size-large">
<img
src="https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2021/10/kleiber_edit.png?w=723&amp;ssl=1"
decoding="async" data-recalc-dims="1" />
<figcaption>Figure 1: The scaling of animal metabolism with body
mass. From Scale.</figcaption>
</figure>

Eyeing Figure [1](#fig-kleiber), you might think that metabolism grows
linearly with mass ... after all, the relation is a straight line. But
notice that both the *x* and *y* axes use logarithmic scales. When you
see a straight line on a double-log plot, the slope of the line actually
translates to an *exponent*. In this case, the slope is 3/4, meaning
metabolic rate scales with the 3/4 power of mass:

::: {style="text-align:center"}
[ \displaystyle \text{metabolic rate} \propto
\text{mass}\^{3/4}]{.katex-eq katex-display="false"}
:::

This metabolic scaling is called [Kleiber's
law](https://en.wikipedia.org/wiki/Kleiber%27s_law){target="_blank"
rel="noopener"}, and it's one of the key elements of West's book.
(Unfortunately, as we'll see below, it's not actually a universal law.)

As the name suggests, *Scale* is full of these scaling relations. On
that front, the book is worth reading for the data alone. I can't think
of another popular science book that presents so much data. All told,
*Scale* packs a whopping 81 figures.

Continuing with the strengths of the book, West's writing is clear and
accessible. And he does a good job of tracing the origins of his ideas.
For instance, it is common knowledge among physicists that when you
scale up the size of an object, its properties change. That's because
the volume of the object (which determines mass) grows faster than its
cross-section area (which determines strength). (I think I first
encountered this idea in Isaac Asimov's [*Fantastic
Voyage*](https://en.wikipedia.org/wiki/Fantastic_Voyage){target="_blank"
rel="noopener"}.) Reading *Scale*, I was surprised to find that this
idea was first articulated by Galileo in the 17th century. (That's a fun
fact I would like to have included in my recent essay, [The Evolution of
'Big'](https://economicsfromthetopdown.com/2021/08/12/the-evolution-of-big-how-sociality-made-life-larger/){target="_blank"
rel="noopener"}.)

Another strength of the book (in my mind) is that West is an unabashed
empiricist. On that note, hidden in one of West's footnotes is this gem
from Albert Einstein:

> Propositions arrived at by purely logical means are completely empty
> as regards reality.

To have merit, Einstein implies, logical propositions must be tested
against evidence. West agrees with Einstein, as do I.

Trained as a physicist, West is at his best when talking about physical
processes. Like most physicists, he identifies the flow of energy as the
root driver of complexity. West is weakest when discussing social
systems. I agree with him that energy is the basis of human society. Yet
I find his worldview a bit too mechanistic. Still, I recommend the book.

### When the universal is ... not universal {#when-the-universal-is-not-universal target="_blank"}

Now to my qualms with *Scale*. The task of writing a popular book on
science is to simplify and generalize, while at the same time not
over-simplifying or over-generalizing. It is a difficult act to balance.
But overall, West does a good job.

Still, there are places where he oversteps. In particular, I find that
West has a bias for 'universality'. West and I agree that a basic part
of doing science is to find similarities that underlie differences.
Indeed, discovering a scaling relation that extends across disparate
entities is exhilarating ... even intoxicating. And for that reason, you
must be careful. 'Universality' is an extremely strong claim that must
be backed by extremely strong evidence. On that front, I think West
overstates his case.

In particular, readers should be aware that West's prime example of a
universal scaling relation --- Kleiber's law, the scaling of metabolic
rate with the 3/4 power of mass --- is not universal. It turns out to be
a quirk of mammals and birds.

In their paper ['Linking scaling laws across
eukaryotes'](https://www.pnas.org/content/116/43/21616){target="_blank"
rel="noopener"}, Ian Hatton and colleagues look at metabolic scaling
across all life on Earth. They find that Kleiber's law does not hold.
Figure [2](#fig-hatton) shows their data relating metabolism to mass
across the whole spectrum of non-plant life. The dashed line shows
Kleiber's law. It works well for mammals and birds. It fails everywhere
else. In fact, the trend that appears 'universal' is a simple linear
scaling of metabolism with mass.

[]{#fig-hatton}

<figure class="wp-block-image size-large">
<img
src="https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2021/10/hatton.png?w=723&amp;ssl=1"
decoding="async" data-recalc-dims="1" />
<figcaption>Figure 2: Metabolic scaling across all non-plant
life. Data is from Hatten et al (2019). The dashed line is
‘Kleiber’s law’ — the scaling of metabolism with the 3/4 power of mass.
It holds for mammals and birds, but nowhere else. The dark line is the
regression across all species, which is nearly linear (the slope is
0.96.)</figcaption>
</figure>

So here we see the perils of looking for 'universality'. West's
generalizations about metabolic scaling come from a small subset of
life. For perspective, the pink square in Figure [2](#fig-hatton) shows
the range of West's data.

Now, this kind of thing happens all the time in science. Conclusions
drawn from limited data often change when the data is enlarged. When
Edwin Hubble first discovered that the universe was expanding, he pegged
the rate at 500 km/s per megaparsec. That's about 7 times higher than
today's accepted rate (which interestingly enough, is [still
debated](https://platosrealm.blog/2020/05/05/the-hubble-tension-a-sign-of-new-physics-in-cosmology/){target="_blank"
rel="noopener"}).

The problem comes not from being wrong, but when wrong theories become
'canonical'. That means they live on, even when the data no longer
supports them. Unfortunately, that may be the case with West's work on
metabolism. His explanation of Kleiber's law has become famous. (He
first wrote about it [2 decades
ago](https://www.astro.princeton.edu/~burrows/classes/542/papers/Lachlan.West_Brown_Enquist_1997.pdf){target="_blank"
rel="noopener"}.) Yet it turns out that Kleiber's law is not actually a
'law' --- it's just a local trend. That puts West's theory in a tough
spot.

### Conflating 'patents' with 'innovation' {#conflating-patents-with-innovation target="_blank"}

Another problem with West's book is that when he turns his attention to
human societies, he uses metrics that I consider dubious.

For instance, West measures 'innovation' in terms of the number of
patents. As Figure [3](#fig-patents) shows, he finds that 'innovation'
scales 'superlinearly' with city population. (For every doubling of city
size, patents *more than* double.) Thus, larger cities seem to be more
innovative.

[]{#fig-patents}

<figure class="wp-block-image size-large">
<img
src="https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2021/10/patents_edit.png?w=723&amp;ssl=1"
decoding="async" data-recalc-dims="1" />
<figcaption>Figure 3: The number of patents scales superlinearly
with city size. From Scale.</figcaption>
</figure>

I dispute the claim that patents measure 'innovation'. What patents
actually measure is the *privatization* of innovation. Patents are the
act of putting property rights around an idea. So they don't measure
innovation so much as they measure the *restriction* of new ideas.

Using patents as a proxy for innovations is similar to measuring the
spread of 'knowledge' using textbook sales. Sure, more textbooks means
more knowledge. But higher textbook sales doesn't necessarily mean that
more textbooks are being published. It could be that publishers are
simply hiking textbook prices. That's not the spread of knowledge.
That's monopolistic firms earning more profits by [restricting access to
knowledge](https://economicsfromthetopdown.com/2021/03/17/why-are-most-textbooks-still-proprietary/){target="_blank"
rel="noopener"}.

With that in mind, we could turn West's claim about innovation on its
head. To echo [Tim Di
Muzio](https://bnarchives.yorku.ca/530/){target="_blank"
rel="noopener"}, every patent is a mini tragedy --- an enclosure of
knowledge that could otherwise be free. So you could argue that it is
not 'innovation' that scales superlinearly with city size, but rather,
the enclosure of the commons. That's not something to celebrate.

### Income, not 'productivity' {#income-not-productivity target="_blank"}

Along the same lines, West shows that income scales superlinearly with
city size. Here's his data for US wages:

[]{#fig-firms}

<figure class="wp-block-image size-large">
<img
src="https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2021/10/wages_edit.png?w=723&amp;ssl=1"
decoding="async" data-recalc-dims="1" />
<figcaption>Figure 4: US wages scale superlinearly with city
size. From Scale.</figcaption>
</figure>

That's interesting. The data tells us that income per capita increases
with city size. This is actually a well-known result in urbanism
literature. There is almost always an urban-rural income gap. So West
generalizes this gap, which is an important thing to do. But then he
does something dubious. He claims that this income data shows that
productivity increases with city population.

No!

To be fair to West, he is making a mistake that is rife in economics:
conflating income with productivity. The mistake dates to the
19th-century work of John Bates Clark, the founder of marginal
productivity theory. In the introduction to his book *The Distribution
of Wealth*, Clark declared that if the market worked "without friction"
it would "give to every agent of production the amount of wealth which
that agent creates." Since then, the standard practice in economics is
to assume that income and productivity are the same thing.

And yet they are not.

To measure productivity, you must differentiate between the quantity of
what is produced and the price of this product. Unfortunately, in all
but the most trivial examples, you [can't actually make this
distinction](https://economicsfromthetopdown.com/2019/11/14/productivity-does-not-explain-wages/){target="_blank"
rel="noopener"}. So the claim that income reveals productivity is an
[empty
tautology](https://economicsfromthetopdown.com/2019/07/08/no-productivity-does-not-explain-income/){target="_blank"
rel="noopener"}. I'm sad that West falls into it.

That said, West's income-scaling evidence is important, and it deserves
an explanation. What West does not consider is that there is a power
dynamic between urban and rural places. Cities are resource-sucking
vacuums that, by definition, cannot support their own consumption. So it
follows that they must have the power to centralize resources. Could it
be that the scaling of income with city size shows this centralization
power? Perhaps not, but one should at least consider the possibility.

Or what about the fact that rural folks tend to do more activities
outside the market --- a fact that would depress income. I grew up in a
[small
town](https://en.wikipedia.org/wiki/Stettler%2C_Alberta){target="_blank"
rel="noopener"} and never paid for a haircut until I moved to the city
at 18. Prior to that, my mom cut my hair for free. That kind of thing
happens less in the city. In other words, city life comes with
specialization, and specialization lends itself to more market
transactions, and more income. None of this has to do with productivity,
at least not directly.

### Firm foibles {#firm-foibles target="_blank"}

Another area where West overstates his case is with firms. Firms, West
argues, seem to be different than cities in that their various forms of
income scale sublinearly with size (employment). Figure [4](#fig-firms)
shows his data. The bracketed numbers show the scaling exponents, which
are all less than one, indicating sublinear scaling.

[]{#fig-firms}

<figure class="wp-block-image size-large">
<img
src="https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2021/10/firm_scale_edit.png?w=723&amp;ssl=1"
decoding="async" data-recalc-dims="1" />
<figcaption>Figure 4: Firm income scales sublinearly with
employment. From Scale.</figcaption>
</figure>

The problem is that West is drawing conclusions from a very narrow
sample of companies. He's using the Compustat database, which only
includes data for public firms. But these public firms are a tiny sample
of the firm universe --- a sample that is highly skewed towards large,
profitable firms.

In effect, West is doing the same thing that he did with animal
metabolism: drawing sweeping conclusions from limited data. If West
wants to draw conclusions about scaling 'laws' amongst firms, then he
needs a representative dataset --- one that includes the vast universe
of small private firms. I'm guessing that if we included these firms,
the scaling relations would change significantly.

### Firm survivorship {#firm-survivorship target="_blank"}

For one last example of West overstating universality, take the survival
of firms over time. The figure below shows West's data, which
demonstrates a remarkable similarity between firms of various size. They
all 'die off' at a similar rate.

[]{#fig-survive}

<figure class="wp-block-image size-large">
<img
src="https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2021/10/firm_survive_edit.png?w=723&amp;ssl=1"
decoding="async" data-recalc-dims="1" />
<figcaption>Figure 6: Firm survival over time. From
Scale.</figcaption>
</figure>

Of course, this data is interesting. But it doesn't mean firm dynamics
are the same everywhere. In fact they are not.

My [own
research](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0171823){target="_blank"
rel="noopener"} has shown that firm survival tends to vary by energy
consumption. As Figure [7](#fig-age-energy) demonstrates, societies that
use less energy per capita tend to have more young firms. So here we
have another 'scaling' relation, but one that emphasizes difference
rather than sameness.

[]{#fig-age-energy}

<figure class="wp-block-image size-large">
<img
src="https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2021/10/firm_age.png?w=723&amp;ssl=1"
decoding="async" data-recalc-dims="1" />
<figcaption>Figure 7: As societies consume more energy, the
portion of young firms tends to decrease. Data is from <a
href="https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0171823"
target="_blank" rel="noopener">‘Energy and Institution
Size’</a>.</figcaption>
</figure>

### Yes, read *Scale*. No, don't believe everything it says. {#yes-read-scale.-no-dont-believe-everything-it-says. target="_blank"}

Overall, my thoughts about *Scale* are similar to my thoughts about
Thomas Piketty's book *Capital in the Twenty-First Century*. Both books
are worth reading for the rich trove of data they present. And both
books make theoretical claims that are (in my mind) dubious.

To be fair, it is part of a scientist's job to convince others that
their ideas are correct. The tools for doing so are evidence (which West
provides in droves) and good arguments. My worry is that West's
arguments (which are admittedly convincing) are more sweeping than the
data warrants.

In a sense, this is a problem with all popular science books. If they
are too convincing, they eventually outlive their usefulness. The ideas
in a book are forever static. Yet science changes with time. Thus, the
continued popularity of books like Marx's *Capital* or Dawkin's *Selfish
Gene* is more of a burden than a boon. These books were long ago made
irrelevant by new research.

On that note, a major part of West's book --- his arguments about
metabolic scaling --- is in danger of becoming irrelevant (or at best,
far less universal than West claims). Two years after *Scale* was
published, Hatton's 2019 study decimated West's theory. But given the
popularity of West's writing, I suspect that his theory will live on for
a long time, evidence be damned.

Problems aside, there is much to like about *Scale*, and it is certainly
a major contribution to science. Just don't believe everything you read
in it.

------------------------------------------------------------------------

#### Support this blog

Economics from the Top Down is where I share my ideas for how to create
a better economics. If you liked this post, consider becoming a patron.
You'll help me continue my research, and continue to share it with
readers like you.

[![patron_button](https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2023/05/supporter_button-1.png?w=220&ssl=1){.aligncenter
decoding="async"
recalc-dims="1"}](https://economicsfromthetopdown.com/membership/){target="_blank"
rel="noopener"}

------------------------------------------------------------------------

#### Stay updated

Sign up to get email updates from this blog.

:::: {.jetpack_subscription_widget}
## {#section .widgettitle}

::: {.wp-block-jetpack-subscriptions__container}
Email Address

Keep me up to date
:::
::::

------------------------------------------------------------------------

[![](https://economicsfromthetopdown.files.wordpress.com/2021/03/by.png?w=150){.aligncenter
decoding="async"
recalc-dims="1"}](http://creativecommons.org/licenses/by/4.0/){rel="license"} 
This work is licensed under a [Creative Commons Attribution 4.0
License](http://creativecommons.org/licenses/by/4.0/){rel="license"}.
You can use/share it anyway you want, provided you attribute it to me
(Blair Fix) and link to [Economics from the Top
Down](https://economicsfromthetopdown.com/).

------------------------------------------------------------------------

### Further reading

Di Muzio, T. (2017). *The tragedy of human development. A genealogy of
capital as power*. Rowman & Littlefield International.

Fix, B. (2017). Energy and institution size. *PLOS ONE*, *12*(2),
e0171823.

Hatton, I. A., Dobson, A. P., Storch, D., Galbraith, E. D., & Loreau, M.
(2019). Linking scaling laws across eukaryotes. *Proceedings of the
National Academy of Sciences*, *116*(43), 21616--21622.

Piketty, T. (2014). *Capital in the twenty-first century*. Cambridge:
Harvard University Press.

West, G. B. (2017). *Scale: The universal laws of growth, innovation,
sustainability, and the pace of life in organisms, cities, economies,
and companies*. Penguin.

West, G. B., Brown, J. H., & Enquist, B. J. (1997). A general model for
the origin of allometric scaling laws in biology. *Science*,
*276*(5309), 122--126.

For several years, people have been urging me to read Geoffrey West’s book

 Scale

, claiming that his work overlaps with my own. Having read the book, I largely agree (with some caveats below). Geoffrey West is a physicist who’s become famous for studying ‘scaling’ effects across life on Earth. What is ‘scaling’? It’s best explained through an example. Figure 1 (taken from West’s book) shows how animals’ metabolism varies with body mass.

A Review of Geoffrey West’s ‘Scale’

My thoughts on the state of corporate power in the US.

I recently had a lively [Twitter
debate](https://twitter.com/blair_fix/status/1442175886569181193){target="_blank"
rel="noopener"} with Jonathan Nitzan, Shimshon Bichler and Cory
Doctorow[^1^](#fn1){#fnref1 .footnote-ref} about the future of big
corporations in the United States. The debate was prompted by Doctorow's
piece ['End of the line for
Reaganomics'](https://capitalaspower.com/2021/09/end-of-the-line-for-reaganomics/){target="_blank"
rel="noopener"}, which I reposted on
[capitalaspower.com](https://capitalaspower.com/){target="_blank"
rel="noopener"}.

Doctorow argues that we may be witnessing a sea change in the way
governments treat big corporations. Since the Reagan era, the US
government has taken most of the teeth out of antitrust enforcement. The
reason is not well known. In fact, I'm ashamed to admit that as a
trained political economist, I didn't learn this antitrust history in
grad school. I learned it from Doctorow's blog.

### In Bork we trust {#in-bork-we-trust target="_blank"}

The antitrust story revolves around a judge named [Robert
Bork](https://en.wikipedia.org/wiki/Robert_Bork){target="_blank"
rel="noopener"}, who came up with a way to defang antitrust law by
changing how it was interpreted. His 1978 book [*The Antitrust
Paradox*](https://en.wikipedia.org/wiki/The_Antitrust_Paradox){target="_blank"
rel="noopener"} argued that antitrust law should be interpreted narrowly
in terms of 'consumer welfare'. And 'consumer welfare', in turn, meant
one thing: *low prices*.

To make an antitrust case, Bork argued that you needed to show that the
offending firm had used its monopoly power to raise prices. Moreover,
you needed to demonstrate that government intervention would do more
good than harm. The paradox, according to Bork, was that by interfering
in the 'free market', antitrust prosecution tended to protect
inefficient firms from competition, and so led to higher prices.

From a scientific standpoint, Bork's arguments are a flaming pile of
garbage. But they are fiendishly clever ideology. Once judges were
indoctrinated in the Borkian worldview, it became nearly impossible to
successfully prosecute an antitrust case. The problem is simple: if a
sector is monopolized, you cannot tell what the prices would be if the
sector was *not* monopolized.

Take, for example, big tech companies like Google and Facebook, which
[dominate](https://martech.org/almost-70-of-digital-ad-spending-going-to-google-facebook-amazon-says-analyst-firm/){target="_blank"
rel="noopener"} the market for online advertisements. Do these companies
use their power to set prices? Absolutely. Can we tell what advertising
prices would be in a 'competitive' market? Absolutely not.

Sure, we can speculate about these prices. We can even run models. But
once we start doing that, antitrust law devolves into an obscure
exercise in model building. In other words, it enters the terrain of
neoclassical economics, where absurd assumptions get buried in a wall of
math, and where the outcome can be tweaked to give the results you want.
Whoever can hire the most model-building economists wins. When you play
that game, deep-pocketed corporations are almost always victorious. And
so as Bork's ideas became widespread (in the 1980s), antitrust law all
but died.

Fortunately, Doctorow notes, there are signs that things are changing.
The Biden administration has put [antitrust proponent Lina
Khan](https://www.yalelawjournal.org/pdf/e.710.Khan.805_zuvfyyeh.pdf){target="_blank"
rel="noopener"} in charge of running the FTC. Under her watch, the
government has launched a major antitrust case against Facebook.

That's the good news. The bad news is that the judiciary remains
indoctrinated in the Borkian worldview. In June 2021, justice James E.
Boasberg [threw
out](https://www.nytimes.com/2021/06/28/technology/facebook-ftc-lawsuit.html){target="_blank"
rel="noopener"} the Facebook case, arguing that prosecutors had failed
to provide evidence that Facebook was a monopolist. Channelling Borkian
ideology, Boasberg
[wrote](https://law.justia.com/cases/federal/district-courts/district-of-columbia/dcdce/1:2020cv03590/224921/73/){target="_blank"
rel="noopener"}:

> It is almost as if the agency expects the Court to simply nod to the
> conventional wisdom that Facebook is a monopolist. After all, no one
> who hears the title of the 2010 film "The Social Network" wonders
> which company it is about. Yet, whatever it may mean to the public,
> *'monopoly power' is a term of art under federal law with a precise
> economic meaning*.
>
> ::: {style="text-align:right" target="_blank"}
> (my emphasis)
> :::

What Boasberg actually means is that under federal law, 'monopoly power'
has a precise *ideological* meaning that differs from the common-sense
view held by the public. Hence, what seems like an obvious monopoly to
the average person is, in the eyes of the law, a 'competitive market'.

Legal setbacks aside, Doctorow remains optimistic that antitrust law is
poised to regain its teeth.

### A tight grip on power {#a-tight-grip-on-power target="_blank"}

Alright, that's Doctorow's take. Jonathan Nitzan and Shimshon Bichler
then entered the [Twitter
debate](https://twitter.com/blair_fix/status/1442175886569181193){target="_blank"
rel="noopener"} backed by their own research on corporate power. For
their part, Nitzan and Bichler are less optimistic that the US
government will seriously challenge the power of big corporations.

Here's their reasoning. By several metrics, corporate power has tracked
upwards over the last 70 years. Figure [1](#fig-bn-profit) shows Bichler
and Nitzan's analysis of profit concentration in the United States. The
blue line shows the share of profits earned by the top 100 US
corporations (ranked by market capitalization). The red line shows the
profit share of the top 500 US corporations. Both measures of
concentration have trended upward since data began in 1950.

[]{#fig-bn-profit}

<figure class="wp-block-image size-large">
<img
src="https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2021/09/bn_profit_share.png?w=723&amp;ssl=1"
decoding="async" data-recalc-dims="1" />
<figcaption>Figure 1: The largest US corporations’ share of
total profit. Source: <a
href="https://rwer.wordpress.com/2021/02/05/dominant-capital-is-much-more-powerful-than-you-think/"
target="_blank" rel="noopener">Bichler and Nitzan</a>.</figcaption>
</figure>

While this trend is worrying, Bichler and Nitzan argue that it actually
understates the growth of corporate power. That's because at the same
time that big corporations have become more profitably, small
corporations have struggled. So relative to the *average* level of
profit, big corporations' grip on power has grown to astonishing levels.

Figure [2](#fig-bn-dif-profit) shows this alternative measure of
corporate power. The blue line compares the average profits of the top
100 US corporation (ranked by market capitalization) to the average
profits of all US corporations. The red line shows the same ratio, but
measures the profits of the top 500 corporations. Both metrics have
increased by more than an order of magnitude since 1950.

[]{#fig-bn-dif-profit}

<figure class="wp-block-image size-large">
<img
src="https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2021/09/bn_differential_profit.png?w=723&amp;ssl=1"
decoding="async" data-recalc-dims="1" />
<figcaption>Figure 2: Profits of top US corporations relative to
the average profit of all US corporations. Source: <a
href="https://rwer.wordpress.com/2021/02/05/dominant-capital-is-much-more-powerful-than-you-think/"
target="_blank" rel="noopener">Bichler and Nitzan</a>.</figcaption>
</figure>

Looking at this data, Bichler and Nitzan argue that big corporations are
more powerful than ever. So although the Biden administration may talk
about enforcing antitrust law, Bichler and Nitzan think that the
government is unlikely to put a serious dent in the power of dominant
firms.

### Big corporations vs. big government {#big-corporations-vs.-big-government target="_blank"}

Here's where I enter the debate. I am more optimistic than Bichler and
Nitzan that the US government will challenge corporate power. The reason
is that the COVID crisis has forced a drastic change in government
policy. Let's have a look.

What I think is missing from Bichler and Nitzan's analysis is government
itself. Their data shows that when judged against the average company,
top corporations are shockingly powerful. What this data does *not*
show, however, is that big corporations have government under their
thumb. To make the case for the corporate dominance of government, we
need to analyze government behavior.

One option is to look at government policy and determine whether it is
pro-big-business or not. That's a messy job that I'll leave for a later
date.

The other option is to simply measure the size of government and see how
it compares to the size of big corporations. This size gives us a sense
of the tug-of-war between the public and private sector. It's not a
perfect indicator, since government spending can be channelled into
private-sector coffers (that's what the Pentagon does). Still, when the
government spends money, it is using its power to shape society.

With that in mind, I'm going to compare the sales of top US corporations
to the expenditure of the US federal government.[^2^](#fn2){#fnref2
.footnote-ref} Think of this comparison as a measure of corporate power
relative to government power. If big US corporations have government
under their thumb, we expect this metric to increase. And indeed it has.

Figure [3](#fig-gov) shows trend in the United States. I've plotted here
the ratio between the sales of the top 500 US firms (ranked by sales)
and the expenditure of the US federal government. From 1950, corporate
power trended upwards, as indicated by the dashed grey line.

[]{#fig-gov}

<figure class="wp-block-image size-large">
<img
src="https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2021/09/gov_plot_2.png?w=723&amp;ssl=1"
decoding="async" data-recalc-dims="1" />
<figcaption>Figure 3: Sales of the top 500 US corporations
relative to expenditures of the Federal government. I’ve ranked
corporations by sales, and then aggregated the sales of the 500 largest
corporations. Then I divide these sales by US federal expenditures. [<a
href="#sources-and-methods" target="_blank" rel="noopener">Sources and
methods</a>].</figcaption>
</figure>

The steady creep of corporate power (from 1950 to 2019) elicits a
feeling of inevitability. And had 2020 been a 'normal' year, corporate
power might have continued to increase. But 2020 was not a normal year.
It was the year of COVID. While there is little to love about the COVID
pandemic, it did demonstrate something important: *society is the way we
make it*.

It is our collective ethos that determines the balance of power between
big corporations and government. Yes, the ethos can seem inevitable. And
under 'normal' circumstance, it is difficult to change. That's why for
the last half century, big corporations increasingly dominated society,
all the while championing the ethos of ['free
markets'](https://economicsfromthetopdown.com/2021/09/16/the-long-slow-death-of-the-free-market/){target="_blank"
rel="noopener"}. But we should not forget that an ethos can change,
sometimes in a shockingly short time.

And that, I believe, is what we are witnessing in Figure [3](#fig-gov).
In a single year --- 2020 --- the corporate grip on power was reversed
to levels not seen since 1952. We all know what happened. Faced with the
need to keep people home but not have them starve, governments around
the world went on spending sprees so massive that they would make JM
Keynes blush.

While we can debate the effectiveness of this spending, what seems clear
is that it constitutes a cataclysmic shift in our collective ethos. Talk
about free-market competition was suddenly dominated by talk about
social welfare. As David Graeber put it, COVID woke us from a collective
dream:

> \[t\]he crisis we just experienced was waking from a dream, a
> confrontation with the actual reality of human life, which is that we
> are a collection of fragile beings taking care of one another, and
> that those who do the lion's share of this care work that keeps us
> alive are overtaxed, underpaid, and daily humiliated, and that a very
> large proportion of the population don't do anything at all but spin
> fantasies, extract rents, and generally get in the way of those who
> are making, fixing, moving, and transporting things, or tending to the
> needs of other living beings.
>
> ::: {style="text-align:right" target="_blank"}
> ([David Graeber,
> 2021](https://jacobinmag.com/2021/03/david-graeber-posthumous-essay-pandemic){target="_blank"
> rel="noopener"})
> :::

My take, then, is that Doctorow's optimism is justified. Yes, Bichler
and Nitzan are correct that big corporations became increasingly
powerful over the last half century --- a period during which the public
sector was beaten into submission by private power. And until 2020, the
growth of corporate power looked inevitable. But it was not. The last
year has shown that society is what we make it. If we want pro-social
spending that shirks the doctrine of free markets, we can have it.

The last year has been surreal in so many ways. Stock markets [are
up](https://economicsfromthetopdown.com/2020/09/04/stocks-are-up-wages-are-down-what-does-it-mean/){target="_blank"
rel="noopener"}. Unemployment reached
[levels](https://fred.stlouisfed.org/series/UNRATE/){target="_blank"
rel="noopener"} not seen since the Great Depression. And yet the poverty
rate [actually
fell](https://www.nytimes.com/2021/09/14/business/economy/census-income-poverty-health-insurance.html){target="_blank"
rel="noopener"}. That's the funny thing about giving people money. It
makes them unpoor.

Moreover, widespread eviction moratoriums relieved the burden of paying
rent. By so doing, they laid bare the facts of property rights. There is
no 'natural law' that says landlords should earn money from their
property. No, that stems from the human-made law of property rights. And
guess what ... if the government stops enforcing that law, property
income vanishes.

Whether all the changes that COVID has wrought will last is anyone's
guess. And whether these changes will translate into robust antitrust
policy is unknown. But I prefer to remain hopeful that pro-social change
is possible.

------------------------------------------------------------------------

#### Support this blog

Economics from the Top Down is where I share my ideas for how to create
a better economics. If you liked this post, consider becoming a patron.
You'll help me continue my research, and continue to share it with
readers like you.

[![patron_button](https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2023/05/supporter_button-1.png?w=220&ssl=1){.aligncenter
decoding="async"
recalc-dims="1"}](https://economicsfromthetopdown.com/membership/){target="_blank"
rel="noopener"}

------------------------------------------------------------------------

#### Stay updated

Sign up to get email updates from this blog.

:::: {.jetpack_subscription_widget}
## {#section .widgettitle}

::: {.wp-block-jetpack-subscriptions__container}
Email Address

Keep me up to date
:::
::::

------------------------------------------------------------------------

[![](https://economicsfromthetopdown.files.wordpress.com/2021/03/by.png?w=150){.aligncenter
decoding="async"
recalc-dims="1"}](http://creativecommons.org/licenses/by/4.0/){rel="license"} 
This work is licensed under a [Creative Commons Attribution 4.0
License](http://creativecommons.org/licenses/by/4.0/){rel="license"}.
You can use/share it anyway you want, provided you attribute it to me
(Blair Fix) and link to [Economics from the Top
Down](https://economicsfromthetopdown.com/).

------------------------------------------------------------------------

### Sources and methods {#sources-and-methods target="_blank"}

For details about Bichler and Nitzan's analysis, see their research
note: ['Dominant capital is much more powerful than you
think'](https://rwer.wordpress.com/2021/02/05/dominant-capital-is-much-more-powerful-than-you-think/){target="_blank"
rel="noopener"}.

Data for the sales of top US corporations is from COMPUSTAT, series
`SALES`. I rank firms by sales, and then sum the gross income of the top
500 firms. Data for US federal government expenditures comes from FRED
series `FGEXPND`.

### Notes {#notes target="_blank"}

::: {.footnotes target="_blank"}
1. [Doctorow is a science fiction author by trade, but he's also a
 prolific blogger who writes about the perils of big tech, monopoly,
 and the abuses of property rights. (Check out his blog
 [Pluralistic](https://pluralistic.net/){target="_blank"
 rel="noopener"}). I'm an avid reader of Doctorow's work, and find
 that he often writes more insightful commentary about modern
 capitalism than do trained economists. The reason is painfully
 obvious to heretics like me: Doctorow hasn't been brainwashed by the
 economics profession, so he retains his critical
 faculties.[↩](#fnref1){.footnote-back}]{#fn1}
2. [A note on my analysis of corporate-vs-government power. On the
 corporate side, I use sales (not profit) to measure firm size
 because there is no equivalent of profit in the public sector. And
 on the government size, I use expenditures (rather than tax revenue)
 because tax revenue is only part of the government's 'income'. The
 federal government can also create its own income by printing money.
 So total expenditures tells us about both taxation and money
 creation.[↩](#fnref2){.footnote-back}]{#fn2}
:::

I recently had a lively Twitter debate with Jonathan Nitzan, Shimshon Bichler and Cory Doctorow

 1

about the future of big corporations in the United States. The debate was prompted by Doctorow’s piece ‘End of the line for Reaganomics’, which I reposted on capitalaspower.com. Doctorow argues that we may be witnessing a sea change in the way governments treat big corporations.

How Dominant are Big US Corporations?

Tim Di Muzio explores the concept of ‘sabotage’ as it applies to capitalist power.

In a series of essays published in 2013 and 2014 on capitaspower.com, political economist Tim Di Muzio explored the concept of ‘sabotage’ as it applies to capitalist power. I recently rediscovered these essays and was so impressed by them that I have reposted them here as a single piece. About the author: Tim Di Muzio is a researcher at the University of Wollongong.

Di Muzio on ‘Sabotage’

Some philosophical thoughts on Canada’s recent election.

Yesterday, Canada had an election that was notable for its
pointlessness. The Liberal party went into the election with a minority
government. And it came out with the same thing.

The election was, however, an interesting testament to the pragmatism of
the powerful. Like Barack Obama, Canadian Prime Minister Justin Trudeau
is gifted at lofty rhetoric. But he's ultimately a pragmatist, dedicated
to holding on to power.

Trudeau called the election because polls suggested that he could win a
majority. Interestingly, when Trudeau looked at the polls, it seems he
didn't factor in the backlash that would result from calling an election
during COVID, especially since the last Canadian election was just 2
years ago. And so Canadians woke up this morning to find the government
largely unchanged.

On that note, the election results remind me of another aspect of power
over principles. Back in 2015, Trudeau ran on a bold platform of
electoral reform, promising to get rid of Canada's archaic
first-past-the-post system.

For a while, it looked like the Trudeau government might make good on
its promise. In 2016, it commissioned a Special Committee on Electoral
Reform that recommended, among other things, that Canada have a
referendum on implementing proportional representation. But the Trudeau
government quietly shelved the report and
[abandoned](https://www.theglobeandmail.com/news/politics/trudeau-abandons-electoral-reform/article33855925/){target="_blank"
rel="noopener"} its commitment to electoral reform.

It's easy to understand this reversal. During the Harper years, the
Liberals suffered a series of humiliating electoral defeats, despite the
fact that their popular support remained largely unchanged. Today, the
tables have turned. The Liberals are now benefiting from the
first-past-the-post system. And so unsurprisingly, they want to keep it.

The numbers speak for themselves. If we take the portion of
parliamentary seats won by each party and divide it them by the portion
of the popular vote, we find that Liberals are hugely over-represented.
Conservatives (who won the popular vote) are more-or-less at parity. And
the NDP and Greens are vastly under-represented.

[]{#fig-elec}

<figure class="wp-block-image size-large">
<img
src="https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2021/09/elec_plot.png?w=723&amp;ssl=1"
decoding="async" data-recalc-dims="1" />
<figcaption>Figure 1: Preliminary results for Canada’s 2021
election. Data is from the <a
href="https://newsinteractives.cbc.ca/elections/federal/2021/results/"
target="_blank" rel="noopener">CBC</a>.</figcaption>
</figure>

During the recent election, Trudeau
[claimed](https://www.ctvnews.ca/politics/federal-election-2021/trudeau-says-he-remains-open-to-electoral-reform-if-liberals-re-elected-1.5590835){target="_blank"
rel="noopener"} to remain 'open' to electoral reform. But given the
numbers above, I suspect this is empty rhetoric. The Liberal's grasp on
power depends on over-representation, which electoral reform would take
away.

In a sense, this dilemma cuts to the core of democracy. We expect the
powerful to do what's in their own best interest. The purpose of
democracy is, in large part, to get the powerful to do things that are
in the *public* interest. Every now and then, representative democracy
succeeds in achieving this inversion. But most of the time it fails.

------------------------------------------------------------------------

#### Support this blog

Economics from the Top Down is where I share my ideas for how to create
a better economics. If you liked this post, consider becoming a patron.
You'll help me continue my research, and continue to share it with
readers like you.

[![patron_button](https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2020/02/patron_button.png?w=220&ssl=1){.aligncenter
decoding="async"
recalc-dims="1"}](https://economicsfromthetopdown.com/membership/){target="_blank"
rel="noopener"}

------------------------------------------------------------------------

#### Stay updated

Sign up to get email updates from this blog.

:::: {.jetpack_subscription_widget}
## {#section .widgettitle}

::: {.wp-block-jetpack-subscriptions__container}
Email Address

Keep me up to date
:::
::::

------------------------------------------------------------------------

[![](https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2021/03/by.png?w=150&ssl=1){.aligncenter
decoding="async"
recalc-dims="1"}](http://creativecommons.org/licenses/by/4.0/){rel="license"} 
This work is licensed under a [Creative Commons Attribution 4.0
License](http://creativecommons.org/licenses/by/4.0/){rel="license"}.
You can use/share it anyway you want, provided you attribute it to me
(Blair Fix) and link to [Economics from the Top
Down](https://economicsfromthetopdown.com/).

------------------------------------------------------------------------

Yesterday, Canada had an election that was notable for its pointlessness. The Liberal party went into the election with a minority government. And it came out with the same thing. The election was, however, an interesting testament to the pragmatism of the powerful. Like Barack Obama, Canadian Prime Minister Justin Trudeau is gifted at lofty rhetoric. But he’s ultimately a pragmatist, dedicated to holding on to power.

Power Over Principles

Tidyquant is a nifty R package for get stock-price data.

They say that knowledge is power. Well, in capitalism, knowledge is also
money ... but only if you keep it under lock and key. Hence, researchers
who study capitalism are often faced with a dilemma: the best data is
kept behind a hefty paywall. The only way to get access is if your
institution pays for a subscription. And if you're not lucky enough to
work for a university, you're out of luck.

Given this problem, I'm always on the look out for open-access data, and
for tools that facilitate access to said data. This week I discovered an
R package called
['Tidyquant'](https://www.rdocumentation.org/packages/tidyquant/versions/1.0.3){target="_blank"
rel="noopener"} that does a nice job of giving access to stock market
data (which it pulls from Yahoo Finance).

Here's a brief tutorial. Install Tidyquant with this command:

 install.packages('tidyquant')

Then load it:

 library(tidyquant)

Next, find the stock ticker for the company you want to look up. As an
example, let's do Exxon Mobile, who's ticker is XOM. The code below will
pull daily Exxon stock prices from the beginning of 1990 to today.

 getSymbols("XOM", 
 from = '1990-01-01',
 to = today(),
 warnings = FALSE,
 auto.assign = TRUE
 )

The function returns a table called `XOM` that looks like this:

<figure class="wp-block-image size-large">
<img
src="https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2021/09/xom_table.png?w=723&amp;ssl=1"
decoding="async" data-recalc-dims="1" />
</figure>

That's pretty nifty. Let's plot some of the data. To plot the closing
price, we enter:

 plot(XOM$XOM.Close)

Here's what the plot looks like. It's not something I'd publish. But
it's good enough to explore the data.

<figure class="wp-block-image size-large">
<img
src="https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2021/09/xom_close.png?w=723&amp;ssl=1"
decoding="async" data-recalc-dims="1" />
</figure>

Suppose we want to compare Exxon's stock price to the S&P 500. That's
easy enough. On Yahoo Finance, the S&P 500 symbol is \^GSPC. Plug that
into our function:

 getSymbols("^GSPC", 
 from = '1990-01-01',
 to = today(),
 warnings = FALSE,
 auto.assign = TRUE
 )

You'll get a table called `GSPC` containing daily data for S&P 500
prices.

<figure class="wp-block-image size-large">
<img
src="https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2021/09/gspc_table.png?w=723&amp;ssl=1"
decoding="async" data-recalc-dims="1" />
</figure>

Next, let's do some simple differential analysis. Let's see how Exxon's
stock price has changed relative to the S&P 500. That's another one
liner:

 plot(XOM$XOM.Close / GSPC$GSPC.Close)

Here's the result, the ratio of Exxon's price to the S&P 500:

<figure class="wp-block-image size-large">
<img
src="https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2021/09/xom_gspc.png?w=723&amp;ssl=1"
decoding="async" data-recalc-dims="1" />
</figure>

As you can see, Exxon hasn't been doing so well since oil prices peaked
in 2008.

Tidyquant is a nifty little tool that gives you quick access to Yahoo
Finance data. While it's no replacement for the paywalled data that is
closely guarded by Bloomberg and the likes, it's a decent package for
doing exploratory research.

------------------------------------------------------------------------

#### Support this blog

Economics from the Top Down is where I share my ideas for how to create
a better economics. If you liked this post, consider becoming a patron.
You'll help me continue my research, and continue to share it with
readers like you.

[![patron_button](https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2020/02/patron_button.png?w=220&ssl=1){.aligncenter
decoding="async"
recalc-dims="1"}](https://economicsfromthetopdown.com/membership/){target="_blank"
rel="noopener"}

------------------------------------------------------------------------

#### Stay updated

Sign up to get email updates from this blog.

:::: {.jetpack_subscription_widget}
## {#section .widgettitle}

::: {.wp-block-jetpack-subscriptions__container}
Email Address

Keep me up to date
:::
::::

------------------------------------------------------------------------

[![](https://i0.wp.com/economicsfromthetopdown.com/wp-content/uploads/2021/03/by.png?w=150&ssl=1){.aligncenter
decoding="async"
recalc-dims="1"}](http://creativecommons.org/licenses/by/4.0/){rel="license"} 
This work is licensed under a [Creative Commons Attribution 4.0
License](http://creativecommons.org/licenses/by/4.0/){rel="license"}.
You can use/share it anyway you want, provided you attribute it to me
(Blair Fix) and link to [Economics from the Top
Down](https://economicsfromthetopdown.com/).

------------------------------------------------------------------------

They say that knowledge is power. Well, in capitalism, knowledge is also money … but only if you keep it under lock and key. Hence, researchers who study capitalism are often faced with a dilemma: the best data is kept behind a hefty paywall. The only way to get access is if your institution pays for a subscription. And if you’re not lucky enough to work for a university, you’re out of luck.

Postagens de Rogue Scholar

Tips for Web Scraping in R and Python