Chris Baker

### Why care about patents?

**1. Patents play a critical role in incentivizing innovation,
withoutwhich we wouldn't have much of the technology we rely on
everyday**

What does your iPhone, Google's PageRank algorithm, and a
buttersubstitute called Smart Balance all have in common?

::: {.gallery .caption-position-bottom .caption-effect-slide .hover-effect-zoom .hover-transition itemscope="" itemtype="http://schema.org/ImageGallery"}
::: {.box}
<figure itemprop="associatedMedia" itemscope=""
itemtype="http://schema.org/ImageObject">
<div class="img"
style="background-image: url(&#39;/img/blog-images/2017-09-19-patentsview/iphone.png&#39;);">
<img src="/img/blog-images/2017-09-19-patentsview/iphone.png"
itemprop="thumbnail" alt="IPhone" />
</div>
<a href="/img/blog-images/2017-09-19-patentsview/iphone.png"
itemprop="contentUrl"></a>
<figcaption><p>Iphone</p></figcaption>
</figure>
:::

::: {.box}
<figure itemprop="associatedMedia" itemscope=""
itemtype="http://schema.org/ImageObject">
<div class="img"
style="background-image: url(&#39;/img/blog-images/2017-09-19-patentsview/google.jpg&#39;);">
<img src="/img/blog-images/2017-09-19-patentsview/google.jpg"
itemprop="thumbnail" alt="Google homepage screenshot" />
</div>
<a href="/img/blog-images/2017-09-19-patentsview/google.jpg"
itemprop="contentUrl"></a>
<figcaption><p>Google homepage screenshot</p></figcaption>
</figure>
:::

::: {.box}
<figure itemprop="associatedMedia" itemscope=""
itemtype="http://schema.org/ImageObject">
<div class="img"
style="background-image: url(&#39;/img/blog-images/2017-09-19-patentsview/butter.png&#39;);">
<img src="/img/blog-images/2017-09-19-patentsview/butter.png"
itemprop="thumbnail" alt="Butter" />
</div>
<a href="/img/blog-images/2017-09-19-patentsview/butter.png"
itemprop="contentUrl"></a>
<figcaption><p>Butter</p></figcaption>
</figure>
:::
:::

...They all probably wouldn't be here if not for patents. A
patentprovides its owner with the ability to make money off of something
thatthey invented, without having to worry about someone else copying
theirtechnology. Think Apple would spend millions of dollars developing
theiPhone if Samsung could just come along and [rip
itoff](https://www.reuters.com/article/us-apple-samsung-elec-appeal-idUSKCN1271LF)?Probably
not.

**2. Patents offer a great opportunity for data analysis**

There are two primary reasons for this:

- **Patent data is public**. In return for the exclusive right toprofit
  off an invention, an individual/company has to publiclydisclose the
  details of their invention to the rest of the world.[Examples of
  thosedetails](http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&p=1&u=%2Fnetahtml%2FPTO%2Fsearch-bool.html&r=11&f=G&l=50&co1=AND&d=PTXT&s1=dog&OS=dog&RS=dog)include
  the patent's title, abstract, technology classification,assigned
  organizations, etc.
- **Patent data can answer questions that people care about**.Companies
  (especially big ones like IBM and Google) have a vestedinterest in
  extracting insights from patents, and spend a lot oftime/resources
  trying figure out how to best manage theirintellectual property (IP)
  rights. They're plagued by questions like"who should I sell my
  underperforming patents to," "which technologyareas are open to new
  innovations," "what's going to be the next bigthing in the world of
  buttery spreads," etc. Patents offer a way toprovide data-driven
  answers to these questions.

Combined, these two things make patents a prime target for dataanalysis.
However, until recently it was hard to get at the data insidethese
documents. One had to either collect it manually using theofficial
[United States Patent and
TrademarkOffice](https://en.wikipedia.org/wiki/United_States_Patent_and_Trademark_Office)(USPTO)
[searchengine](http://patft.uspto.gov/netahtml/PTO/search-adv.htm), or
figureout a way to download, parse, and model huge XML data dumps.
EnterPatentsView.

### PatentsView and the `patentsview` package

[PatentsView](https://www.patentsview.org/web/#viz/relationships) is
oneof USPTO's new initiatives intended to increase the usability and
valueof patent data. One feature of this project is a publicly
accessible APIthat makes it easy to programmatically interact with the
data. A few ofthe reasons why I like the API (and PatentsView more
generally):

- The API is free (no credential required) and currently doesn'timpose
  rate limits/bandwidth throttling.
- The project offers [bulk downloads of
  patentdata](https://www.patentsview.org/download/) on their website
  (in aflat file format), for those who want to be closest to the data.
- Both the API and the bulk download data contain disambiguatedentities
  such as inventors, assignees, organizations, etc. In otherwords, the
  API will tell you whether it thinks that John Smith onpatent X is the
  same person as John Smith on patent Y. ^[1](#fn:1){.footnote-ref
  role="doc-noteref"}^

The `patentsview` R package is a wrapper around the PatentsView API.
Itcontains a function that acts as a client to the API (`search_pv()`)
aswell as several supporting functions. Full documentation of the
packagecan be found on
its[website](https://docs.ropensci.org/patentsview/index.html).

### Installation

You can install the stable version of `patentsview` from CRAN:

    install.packages("patentsview")

Or get the development version from GitHub:

    if (!require(devtools)) install.packages("devtools")devtools::install_github("ropensci/patentsview")

### Getting started

The package has one main function, `search_pv()`, that makes it easy
tosend requests to the API. There are two parameters to `search_pv()`
thatyou're going to want to think about just about every time you call
it -`query` and `fields`. You tell the API how you want to filter the
patentdata with `query`, and which fields you want to retrieve
with`fields`. ^[2](#fn:2){.footnote-ref role="doc-noteref"}^

#### `query`

Your query has to use the [PatentsView
querylanguage](https://www.patentsview.org/api/query-language.html),
which isa JSON-based syntax that is similar to the one used by Lucene.
You canwrite the query directly and pass it as a string to
`search_pv()`:

    library(patentsview)qry_1 <- '{"_gt":{"patent_year":2007}}'search_pv(query = qry_1, fields = NULL) # This will retrieve a default set of fields#> $data#> #### A list with a single data frame on the patent data level:#>#> List of 1#>  $ patents:'data.frame': 25 obs. of  3 variables:#>   ..$ patent_id    : chr [1:25] "7313829" ...#>   ..$ patent_number: chr [1:25] "7313829" ...#>   ..$ patent_title : chr [1:25] "Sealing device for body suit and sealin"..#>#> $query_results#> #### Distinct entity counts across all downloadable pages of output:#>#> total_patent_count = 100,000

...Or you can use the domain specific language (DSL) provided in
the`patentsview` package to help you write the query:

    qry_2 <- qry_funs$gt(patent_year = 2007) # All DSL functions are in the qry_funs listqry_2 # qry_2 is the same as qry_1#> {"_gt":{"patent_year":2007}}search_pv(query = qry_2)#> $data#> #### A list with a single data frame on the patent data level:#>#> List of 1#>  $ patents:'data.frame': 25 obs. of  3 variables:#>   ..$ patent_id    : chr [1:25] "7313829" ...#>   ..$ patent_number: chr [1:25] "7313829" ...#>   ..$ patent_title : chr [1:25] "Sealing device for body suit and sealin"..#>#> $query_results#> #### Distinct entity counts across all downloadable pages of output:#>#> total_patent_count = 100,000

`qry_1` and `qry_2` will result in the same HTTP call to the API.
Bothqueries search for patents in USPTO that were published after
2007.There are three gotchas to look out for when writing a query:

1.  **Field is queryable.** The API has 7 endpoints (the defaultendpoint
    is "patents"), and each endpoint has its own set of fieldsthat you
    can filter on. *The fields that you can filter on are notnecessarily
    the same as the ones that you can retrieve.* In otherwords, the
    fields that you can include in `query` (e.g.,`patent_year`) are not
    necessarily the same as those that you caninclude in `fields`. To
    see which fields you can query on, look inthe `fieldsdf` data frame
    (`View(patentsview::fieldsdf)`) for fieldsthat have a "y" indicator
    in their `can_query` column for your givenendpoint.
2.  **Correct data type for field.** If you're filtering on a field
    inyour query, you have to make sure that the value you are
    filteringon is consistent with the field's data type. For
    example,`patent_year` has type "integer," so if you pass 2007 as a
    stringthen you're going to get an error (`patent_year = 2007` is
    good,`patent_year = "2007"` is no good). You can find a field's data
    typein the `fieldsdf` data frame.
3.  **Comparison function works with field's data type.** The
    comparisonfunction(s) that you use (e.g., the greater-than function
    shownabove, `qry_funs$gt()`) must be consistent with the field's
    datatype. For example, you can't use the "contains" function on
    fieldsof type "integer" (`qry_funs$contains(patent_year = 2007)`
    willthrow an error). See `?qry_funs` for more details.

In short, use the `fieldsdf` data frame when you write a query and
youshould be fine. Check out the [writing
queriesvignette](https://ropensci.github.io/patentsview/articles/writing-queries.html)for
more details.

#### `fields`

Up until now we have been using the default value for `fields`.
Thisresults in the API giving us some small set of default fields. Let's
seeabout retrieving some more fields:

    search_pv(  query = qry_funs$gt(patent_year = 2007),  fields = c("patent_abstract", "patent_average_processing_time",             "inventor_first_name", "inventor_total_num_patents"))#> $data#> #### A list with a single data frame (with list column(s) inside) on the patent data level:#>#> List of 1#>  $ patents:'data.frame': 25 obs. of  3 variables:#>   ..$ patent_abstract               : chr [1:25] "A sealing device for a"..#>   ..$ patent_average_processing_time: chr [1:25] "1324" ...#>   ..$ inventors                     :List of 25#>#> $query_results#> #### Distinct entity counts across all downloadable pages of output:#>#> total_patent_count = 100,000

The fields that you can retrieve depends on the endpoint that you
arehitting. We've been using the "patents" endpoint thus far, so all
ofthese are
retrievable:`fieldsdf[fieldsdf$endpoint == "patents", "field"]`. You can
also use`get_fields()` to list the retrievable fields for a given
endpoint:

    search_pv(  query = qry_funs$gt(patent_year = 2007),  fields = get_fields(endpoint = "patents", groups = c("patents", "inventors")))#> $data#> #### A list with a single data frame (with list column(s) inside) on the patent data level:#>#> List of 1#>  $ patents:'data.frame': 25 obs. of  31 variables:#>   ..$ patent_abstract                       : chr [1:25] "A sealing devi"..#>   ..$ patent_average_processing_time        : chr [1:25] "1324" ...#>   ..$ patent_date                           : chr [1:25] "2008-01-01" ...#>   ..$ patent_firstnamed_assignee_city       : chr [1:25] "Cambridge" ...#>   ..$ patent_firstnamed_assignee_country    : chr [1:25] "US" ...#>   ..$ patent_firstnamed_assignee_id         : chr [1:25] "b9fc6599e3d60c"..#>   ..$ patent_firstnamed_assignee_latitude   : chr [1:25] "42.3736" ...#>   ..$ patent_firstnamed_assignee_location_id: chr [1:25] "42.3736158|-71"..#>   ..$ patent_firstnamed_assignee_longitude  : chr [1:25] "-71.1097" ...#>   ..$ patent_firstnamed_assignee_state      : chr [1:25] "MA" ...#>   ..$ patent_firstnamed_inventor_city       : chr [1:25] "Lucca" ...#>   ..$ patent_firstnamed_inventor_country    : chr [1:25] "IT" ...#>   ..$ patent_firstnamed_inventor_id         : chr [1:25] "6416028-3" ...#>   ..$ patent_firstnamed_inventor_latitude   : chr [1:25] "43.8376" ...#>   ..$ patent_firstnamed_inventor_location_id: chr [1:25] "43.8376211|10."..#>   ..$ patent_firstnamed_inventor_longitude  : chr [1:25] "10.4951" ...#>   ..$ patent_firstnamed_inventor_state      : chr [1:25] "Tuscany" ...#>   ..$ patent_id                             : chr [1:25] "7313829" ...#>   ..$ patent_kind                           : chr [1:25] "B1" ...#>   ..$ patent_number                         : chr [1:25] "7313829" ...#>   ..$ patent_num_cited_by_us_patents        : chr [1:25] "5" ...#>   ..$ patent_num_claims                     : chr [1:25] "25" ...#>   ..$ patent_num_combined_citations         : chr [1:25] "35" ...#>   ..$ patent_num_foreign_citations          : chr [1:25] "0" ...#>   ..$ patent_num_us_application_citations   : chr [1:25] "0" ...#>   ..$ patent_num_us_patent_citations        : chr [1:25] "35" ...#>   ..$ patent_processing_time                : chr [1:25] "792" ...#>   ..$ patent_title                          : chr [1:25] "Sealing device"..#>   ..$ patent_type                           : chr [1:25] "utility" ...#>   ..$ patent_year                           : chr [1:25] "2008" ...#>   ..$ inventors                             :List of 25#>#> $query_results#> #### Distinct entity counts across all downloadable pages of output:#>#> total_patent_count = 100,000

### Example

Let's look at a quick example of pulling and analyzing patent data.We'll
look at patents from the last ten years that are classified belowthe
[H04L63/00
CPCcode](https://worldwide.espacenet.com/classification#!/CPC=H04L63/02#!/CPC=H04L63/02).Patents
in this area relate to "network architectures or networkcommunication
protocols for separating internal from externaltraffic."
^[3](#fn:3){.footnote-ref role="doc-noteref"}^ CPC codes offer a quick
and dirty way to find patents ofinterest, though getting a sense of
their hierarchy can be tricky.

1.  Download the data



    library(patentsview)# Write a query:query <- with_qfuns( # with_qfuns is basically just: with(qry_funs, ...)  and(    begins(cpc_subgroup_id = 'H04L63/02'),    gte(patent_year = 2007)  ))# Create a list of fields:fields <- c(  c("patent_number", "patent_year"),  get_fields(endpoint = "patents", groups = c("assignees", "cpcs")))# Send HTTP request to API's server:pv_res <- search_pv(query = query, fields = fields, all_pages = TRUE)

1.  See where the patents are coming from (geographically)



    library(leaflet)library(htmltools)library(dplyr)library(tidyr)data <-  pv_res$data$patents %>%    unnest(assignees) %>%    select(assignee_id, assignee_organization, patent_number,           assignee_longitude, assignee_latitude) %>%    group_by_at(vars(-matches("pat"))) %>%    mutate(num_pats = n()) %>%    ungroup() %>%    select(-patent_number) %>%    distinct() %>%    mutate(popup = paste0("<font color='Black'>",                          htmlEscape(assignee_organization), "<br><br>Patents:",                          num_pats, "</font>")) %>%    mutate_at(vars(matches("_l")), as.numeric) %>%    filter(!is.na(assignee_id))leaflet(data) %>%  addProviderTiles(providers$CartoDB.DarkMatterNoLabels) %>%  addCircleMarkers(lng = ~assignee_longitude, lat = ~assignee_latitude,                   popup = ~popup, ~sqrt(num_pats), color = "yellow")

![](/assets/blog-images/2017-09-19-patentsview/unnamed-chunk-9-1.png)

1.  Plot the growth of the field's topics over time



    library(ggplot2)library(RColorBrewer)data <-  pv_res$data$patents %>%    unnest(cpcs) %>%    filter(cpc_subgroup_id != "H04L63/02") %>% # remove patents categorized into only top-level category of H04L63/02    mutate(      title = case_when(        grepl("filtering", .$cpc_subgroup_title, ignore.case = T) ~          "Filtering policies",        .$cpc_subgroup_id %in% c("H04L63/0209", "H04L63/0218") ~          "Architectural arrangements",        grepl("Firewall traversal", .$cpc_subgroup_title, ignore.case = T) ~          "Firewall traversal",        TRUE ~          .$cpc_subgroup_title      )    ) %>%    mutate(title = gsub(".*(?=-)-", "", title, perl = TRUE)) %>%    group_by(title, patent_year) %>%    count() %>%    ungroup() %>%    mutate(patent_year = as.numeric(patent_year))ggplot(data = data) +  geom_smooth(aes(x = patent_year, y = n, colour = title), se = FALSE) +  scale_x_continuous("\nPublication year", limits = c(2007, 2016),                     breaks = 2007:2016) +  scale_y_continuous("Patents\n", limits = c(0, 700)) +  scale_colour_manual("", values = brewer.pal(5, "Set2")) +  theme_bw() + # theme inspired by https://hrbrmstr.github.io/hrbrthemes/  theme(panel.border = element_blank(), axis.ticks = element_blank())

![](/assets/blog-images/2017-09-19-patentsview/unnamed-chunk-10-1.png)

### Learning more

For analysis examples that go into a little more depth, check out
the[data
applicationsvignettes](https://ropensci.github.io/patentsview/articles/citation-networks.html)on
the package's website. If you're just interested in `search_pv()`,there
are[examples](https://ropensci.github.io/patentsview/articles/examples.html)on
the site for that as well. To contribute to the package or report
anissue, check out the [issues page
onGitHub](https://github.com/ropensci/patentsview/issues).

### Acknowledgments

I'd like to thank the package's two reviewers,
[PaulOldham](https://github.com/poldham) and
[VerenaHaunschmid](https://github.com/expectopatronum/), for taking the
time to reviewthe package and providing helpful feedback. I'd also like
to thank[Maëlle Salmon](https://masalmon.eu/) for shepherding the
packagealong the rOpenSci review process, as well
[ScottChamberlain](https://scottchamberlain.info/) and
[StefanieButland](https://twitter.com/stefaniebutland) for their
miscellaneoushelp.

::: {.footnotes role="doc-endnotes"}

------------------------------------------------------------------------

1.  ::: {#fn:1}
    This is both good and bad, as there are errors in the
    disambiguation. The algorithm that is responsible for the
    disambiguation was created by the winner of the [PatentsView
    Inventor Disambiguation Technical
    Workshop](https://www.uspto.gov/about-us/organizational-offices/office-policy-and-international-affairs/patentsview-inventor/). [↩︎](#fnref:1){.footnote-backref
    role="doc-backlink"}
    :::

2.  ::: {#fn:2}
    These two parameters end up getting translated into a MySQL query by
    the API's server, which then gets sent to a back-end database.
    `query` and `fields` are used to create the query's `WHERE` and
    `SELECT` clauses, respectively. [↩︎](#fnref:2){.footnote-backref
    role="doc-backlink"}
    :::

3.  ::: {#fn:3}
    There is a slightly more in-depth definition that says that these
    are patents "related to the (logical) separation of traffic/(sub-)
    networks to achieve protection." [↩︎](#fnref:3){.footnote-backref
    role="doc-backlink"}
    :::
:::

Why care about patents?
<strong>
 1. Patents play a critical role in incentivizing innovation, withoutwhich we wouldn’t have much of the technology we rely on everyday
</strong>
What does your iPhone, Google’s PageRank algorithm, and a buttersubstitute called Smart Balance all have in common? …They all probably wouldn’t be here if not for patents.

Accessing patent data with the patentsview package

I had a pretty ropey day today. Failing for 45 minutes to do some simple algebra. Transformation overnight that didn’t work…again…and just generally being a bit down what with the whole imploding British science funding situation. But in the midst of this we did one very cool and simple experiment, one that worked, and one that actually has some potentially significant implications. The only things is…I can’t tell you about it.

Cameron Neylon

::: {.zemanta-img}
<div>

[![Notebook collection](http://cameronneylon.net/wp-content/uploads/2010/10/2445114424_17c1d9d0b0_m.jpg "Notebook collection"){decoding="async"}](http://www.flickr.com/photos/21093323@N02/2445114424)
:   Image by
    [Dvortygirl](http://www.flickr.com/photos/21093323@N02/2445114424)
    via Flickr

</div>
:::

I had a pretty ropey day today. Failing for 45 minutes to do some simple
algebra. Transformation overnight that didn't work...again...and just
generally being a bit down what with the whole imploding British science
funding situation. But in the midst of this we did one very cool and
simple experiment, one that worked, and one that actually has some
potentially really interesting implications.

The only things is...I can't tell you about it. It's not my project and
the result is potentially patentable. At one level this is
great..."Neylon in idea that might have practical application
shock!"...but as a person who believes in the value of sharing results
and information as fast as is practical it is frustrating.
[Jean-Claude](http://usefulchem.blogspot.com) when talking about [Open
Notebook
Science](http://en.wikipedia.org/wiki/Open_Notebook_Science "Open Notebook Science"){.zem_slink
rel="wikipedia"} has used a description which I think captures the value
here. "If it's not in the notebook it hasn't been done". I can't really
live up to that statement.

I don't have any simple answers to this. The world is not a simple place
to live in in practice. My job is fundamentally about collaborating and
supporting other scientists. I enjoy this but it does mean that a lot of
what I do is really other people's projects -- and mostly ones where I
don't control the publication schedule. The current arrangement of the
international IP system effectively mandates secrecy for as long as can
be managed, exactly the opposite of what it was intended to do. The [ONS
badges](http://onsclaims.wikispaces.com/) developed by Jean-Claude,
[Andrew](http://webapps.oru.edu/facultyplace/view_profile.php?user_id=74),
and [Shirley](http://shirleywho.wordpress.com/) are great but they solve
the communication problem, the one of explaining what I have done, not
the philosophical one, that I can do what I feel I should. And for
reasons too tedious to go into it's not straightforward to put them on
my notebook.

Like I say, life in the real world is complicated. As much as anything
this is simply a marker to say that not everything I do is made public.
I do what I can when I can, with what I can. But it's a long way from
perfect.

But hey, at least I had an idea that worked!

###### Related articles by Zemanta {#related-articles-by-zemanta .zemanta-related-title}

- [\[2b2k\] Scientific transparency vs.
  trust](http://www.hyperorg.com/blogger/2010/08/28/2b2k-scientific-transparency-vs-trust/)
  (hyperorg.com)
- [Friendfeed for Scientific
  Collaboration](http://scienceroll.com/2010/08/12/friendfeed-for-scientific-collaboration/)
  (scienceroll.com)

::: {.zemanta-pixie}
[![Enhanced by
Zemanta](http://img.zemanta.com/zemified_e.png?x-id=1bea065c-2f45-4c8e-b346-34784d93deff){.zemanta-pixie-img
decoding="async"}](http://www.zemanta.com/ "Enhanced by Zemanta"){.zemanta-pixie-a}[]{.zem-script
.more-related .pretty-attribution .paragraph-reblog}
:::


Image by Dvortygirl via Flickr  I had a pretty ropey day today. Failing for 45 minutes to do some simple algebra. Transformation overnight that didn’t work…again…and just generally being a bit down what with the whole imploding British science funding situation. But in the midst of this we did one very cool and simple experiment, one that worked, and one that actually has some potentially really interesting implications.

Messages de Rogue Scholar

Accessing patent data with the patentsview package

The truth…well most of the truth anyway