[How to] Build an API wrapper package in 10 minutes.

Posted on January 20, 2018 by Colin Fay in R bloggers | 0 Comments

[This article was first published on Colin Fay, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)

Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Well… documentation not included (of course).

API are cool. They allow to retrieve data from the web, and if ever
you’re familiar with {httr}, {jsonlite} and packages like these,
you’re able to build requests and retrieve data in a matter of
minutes.

But no worry, if you’re not familiar with http requests and web formats
like html, JSON and such, you can still go and look for a package
wrapper around that specific API: someone had probably been coding it
already.

And, if you want to be that someone that code an API wrapper package,
here’s a short tutorial that will allow you to create it.

Disclaimer: the 10 minutes workflow does not (of course) include the
package documentation.

Find the API

Well, this first step can take a while… but, let’s say we have already
found it, and want to build an package around the french database for
addresses: https://adresse.data.gouv.fr/api/.

Step 0: be sure you have all the packages you’ll need

Run this if you want to be sure you have all the packages we’ll use
here:

<span class="n">install.packages</span><span class="p">(</span><span class="s2">"devtools"</span><span class="p">)</span><span class="w">
</span><span class="n">install.packages</span><span class="p">(</span><span class="s2">"roxygen2"</span><span class="p">)</span><span class="w">
</span><span class="n">install.packages</span><span class="p">(</span><span class="s2">"usethis"</span><span class="p">)</span><span class="w">
</span><span class="n">install.packages</span><span class="p">(</span><span class="s2">"curl"</span><span class="p">)</span><span class="w">
</span><span class="n">install.packages</span><span class="p">(</span><span class="s2">"httr"</span><span class="p">)</span><span class="w">
</span><span class="n">install.packages</span><span class="p">(</span><span class="s2">"jsonlite"</span><span class="p">)</span><span class="w">
</span><span class="n">install.packages</span><span class="p">(</span><span class="s2">"attempt"</span><span class="p">)</span><span class="w">
</span><span class="n">install.packages</span><span class="p">(</span><span class="s2">"purrr"</span><span class="p">)</span><span class="w">
</span><span class="n">devtools</span><span class="o">::</span><span class="n">install_github</span><span class="p">(</span><span class="s2">"r-lib/desc"</span><span class="p">)</span><span class="w">
</span>

Step 1: the project

Create a new project with RStudio, and click on “Create a package with
devtools”.

Step 2: devstuffs

In your terminal, run usethis::use_data_raw().
Open a new R Script, save it into /data_raw as “devstuffs”, or any
other name. Copy and paste into this script:

<span class="n">library</span><span class="p">(</span><span class="n">devtools</span><span class="p">)</span><span class="w">
</span><span class="n">library</span><span class="p">(</span><span class="n">usethis</span><span class="p">)</span><span class="w">
</span><span class="n">library</span><span class="p">(</span><span class="n">desc</span><span class="p">)</span><span class="w">

</span><span class="c1"># Remove default DESC</span><span class="w">
</span><span class="n">unlink</span><span class="p">(</span><span class="s2">"DESCRIPTION"</span><span class="p">)</span><span class="w">
</span><span class="c1"># Create and clean desc</span><span class="w">
</span><span class="n">my_desc</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">description</span><span class="o">$</span><span class="n">new</span><span class="p">(</span><span class="s2">"!new"</span><span class="p">)</span><span class="w">

</span><span class="c1"># Set your package name</span><span class="w">
</span><span class="n">my_desc</span><span class="o">$</span><span class="n">set</span><span class="p">(</span><span class="s2">"Package"</span><span class="p">,</span><span class="w"> </span><span class="s2">"yourpackage"</span><span class="p">)</span><span class="w">

</span><span class="c1">#Set your name</span><span class="w">
</span><span class="n">my_desc</span><span class="o">$</span><span class="n">set</span><span class="p">(</span><span class="s2">"[email protected]"</span><span class="p">,</span><span class="w"> </span><span class="s2">"person('Colin', 'Fay', email = '[email protected]', role = c('cre', 'aut'))"</span><span class="p">)</span><span class="w">

</span><span class="c1"># Remove some author fields</span><span class="w">
</span><span class="n">my_desc</span><span class="o">$</span><span class="n">del</span><span class="p">(</span><span class="s2">"Maintainer"</span><span class="p">)</span><span class="w">

</span><span class="c1"># Set the version</span><span class="w">
</span><span class="n">my_desc</span><span class="o">$</span><span class="n">set_version</span><span class="p">(</span><span class="s2">"0.0.0.9000"</span><span class="p">)</span><span class="w">

</span><span class="c1"># The title of your package</span><span class="w">
</span><span class="n">my_desc</span><span class="o">$</span><span class="n">set</span><span class="p">(</span><span class="n">Title</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"My Supper API Wrapper"</span><span class="p">)</span><span class="w">
</span><span class="c1"># The description of your package</span><span class="w">
</span><span class="n">my_desc</span><span class="o">$</span><span class="n">set</span><span class="p">(</span><span class="n">Description</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"A long description of this super package I'm working on."</span><span class="p">)</span><span class="w">
</span><span class="c1"># The urls</span><span class="w">
</span><span class="n">my_desc</span><span class="o">$</span><span class="n">set</span><span class="p">(</span><span class="s2">"URL"</span><span class="p">,</span><span class="w"> </span><span class="s2">"http://this"</span><span class="p">)</span><span class="w">
</span><span class="n">my_desc</span><span class="o">$</span><span class="n">set</span><span class="p">(</span><span class="s2">"BugReports"</span><span class="p">,</span><span class="w"> </span><span class="s2">"http://that"</span><span class="p">)</span><span class="w">
</span><span class="c1"># Save everyting</span><span class="w">
</span><span class="n">my_desc</span><span class="o">$</span><span class="n">write</span><span class="p">(</span><span class="n">file</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"DESCRIPTION"</span><span class="p">)</span><span class="w">

</span><span class="c1"># If you want to use the MIT licence, code of conduct, and lifecycle badge</span><span class="w">
</span><span class="n">use_mit_license</span><span class="p">(</span><span class="n">name</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"Colin FAY"</span><span class="p">)</span><span class="w">
</span><span class="n">use_code_of_conduct</span><span class="p">()</span><span class="w">
</span><span class="n">use_lifecycle_badge</span><span class="p">(</span><span class="s2">"Experimental"</span><span class="p">)</span><span class="w">
</span><span class="n">use_news_md</span><span class="p">()</span><span class="w">

</span><span class="c1"># Get the dependencies</span><span class="w">
</span><span class="n">use_package</span><span class="p">(</span><span class="s2">"httr"</span><span class="p">)</span><span class="w">
</span><span class="n">use_package</span><span class="p">(</span><span class="s2">"jsonlite"</span><span class="p">)</span><span class="w">
</span><span class="n">use_package</span><span class="p">(</span><span class="s2">"curl"</span><span class="p">)</span><span class="w">
</span><span class="n">use_package</span><span class="p">(</span><span class="s2">"attempt"</span><span class="p">)</span><span class="w">
</span><span class="n">use_package</span><span class="p">(</span><span class="s2">"purrr"</span><span class="p">)</span><span class="w">

</span><span class="c1"># Clean your description</span><span class="w">
</span><span class="n">use_tidy_description</span><span class="p">()</span><span class="w">
</span>

Then, run everything from this script.

Copy and paste at the top of you README.Rmd
:

  [![lifecycle](https://img.shields.io/badge/lifecycle-experimental-orange.svg)](https://www.tidyverse.org/lifecycle/#experimental)

And at the bottom
:

  Please note that this project is released with a [Contributor Code of Conduct](CODE_OF_CONDUCT.md).
  By participating in this project you agree to abide by its terms.

Step 3: get the API url

There are two ways you can request on an API :

By building url: url.and/path/to/the/data
With parameters: url.that/q=this&data=that

This is the case for https://adresse.data.gouv.fr/api/. We’ve got the
base url: http 'https://api-adresse.data.gouv.fr/search/?q=8 bd du port', and search parameters.

Here, the base url is everything before the ?, and the parameters are
the key-value pairs after the
.

Step 4: utils

Before creating the main calling functions, we’ll create some utilitary
functions that will run some tests: is the internet connexion running?
Does the {httr} result return the right http code?

For this, we’ll create a file called utils.R, save it in the R/
folder, and put into it:

<span class="cd">#' @importFrom attempt stop_if_not</span><span class="w">
</span><span class="cd">#' @importFrom curl has_internet</span><span class="w">
</span><span class="n">check_internet</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="k">function</span><span class="p">(){</span><span class="w">
  </span><span class="n">stop_if_not</span><span class="p">(</span><span class="n">.x</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">has_internet</span><span class="p">(),</span><span class="w"> </span><span class="n">msg</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"Please check your internet connexion"</span><span class="p">)</span><span class="w">
</span><span class="p">}</span><span class="w">

</span><span class="cd">#' @importFrom httr status_code</span><span class="w">
</span><span class="n">check_status</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="k">function</span><span class="p">(</span><span class="n">res</span><span class="p">){</span><span class="w">
  </span><span class="n">stop_if_not</span><span class="p">(</span><span class="n">.x</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">status_code</span><span class="p">(</span><span class="n">res</span><span class="p">),</span><span class="w"> 
              </span><span class="n">.p</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">~</span><span class="w"> </span><span class="n">.x</span><span class="w"> </span><span class="o">==</span><span class="w"> </span><span class="m">200</span><span class="p">,</span><span class="w">
              </span><span class="n">msg</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"The API returned an error"</span><span class="p">)</span><span class="w">
</span><span class="p">}</span><span class="w">
</span>

In this same file we’ll also create two objects that will contain the
base API urls:

<span class="n">base_url</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="s2">"https://api-adresse.data.gouv.fr/search/"</span><span class="w">
</span><span class="n">reverse_url</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="s2">"https://api-adresse.data.gouv.fr/reverse/"</span><span class="w">
</span>

Step 5 : the function that will call on the API

To call the API, we’ll use GET function from {httr}. Let’s first try
just this:

<span class="n">httr</span><span class="o">::</span><span class="n">GET</span><span class="p">(</span><span class="n">url</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">base_url</span><span class="p">,</span><span class="w"> </span><span class="n">query</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="n">q</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"Yeaye"</span><span class="p">))</span><span class="w">
</span>

## Response [https://api-adresse.data.gouv.fr/search/?q=Yeaye]
##   Date: 2018-02-04 21:40
##   Status: 200
##   Content-Type: application/json; charset=utf-8
##   Size: 574 B

As you can see, the status is 200. Which is a good sign: no error from
the API.

In the API we have chosen, there are 4 entry points: search, reverse,
and their counterparts with csv. These counterparts work by POSTing a
csv to the API, so let’s forget them for now.

The search entrypoint can take 8 parameters:

q: text search
limit: maximum number of results to return
autocomplete: autocompletion
lat: latitude
lon: longitude
type: type of elements to return (housenumber, street,place,
municipality)
postcode: Postal code
citycode: INSEE code

These are the elements which will be passed as a list in the query
parameter from httr::GET.

Note: if you’re going for an url based request
(url.and/path/to/the/data or url.and/?path=this&to=that), you
don’t need to set the query parameter, you can simply paste the url
with the elements (url <- paste0("url.and/?path=", this, "&to=", that)).

Create a new R script and write the functions used to call the API:

<span class="cd">#' Search the BAN</span><span class="w">
</span><span class="cd">#' </span><span class="w">
</span><span class="cd">#' @param q text search</span><span class="w">
</span><span class="cd">#' @param limit maximum number of results to return </span><span class="w">
</span><span class="cd">#' @param autocomplete autocompletion</span><span class="w">
</span><span class="cd">#' @param lat latitude</span><span class="w">
</span><span class="cd">#' @param lon longitude</span><span class="w">
</span><span class="cd">#' @param type type of elements to return (housenumber, street,place, municipality)</span><span class="w">
</span><span class="cd">#' @param postcode Postal code</span><span class="w">
</span><span class="cd">#' @param citycode INSEE code</span><span class="w">
</span><span class="cd">#'</span><span class="w">
</span><span class="cd">#' @importFrom attempt stop_if_all</span><span class="w">
</span><span class="cd">#' @importFrom purrr compact</span><span class="w">
</span><span class="cd">#' @importFrom jsonlite fromJSON</span><span class="w">
</span><span class="cd">#' @importFrom httr GET</span><span class="w">
</span><span class="cd">#' @export</span><span class="w">
</span><span class="cd">#' @rdname searchban</span><span class="w">
</span><span class="cd">#'</span><span class="w">
</span><span class="cd">#' @return the results from the search</span><span class="w">
</span><span class="cd">#' @examples </span><span class="w">
</span><span class="cd">#' \dontrun{</span><span class="w">
</span><span class="cd">#' search_ban("Rennes")</span><span class="w">
</span><span class="cd">#' reverse_search_ban("48.11", "-1.68")</span><span class="w">
</span><span class="cd">#' }</span><span class="w">

</span><span class="n">search_ban</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="k">function</span><span class="p">(</span><span class="n">q</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">NULL</span><span class="p">,</span><span class="w"> </span><span class="n">limit</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">NULL</span><span class="p">,</span><span class="w"> </span><span class="n">autocomplete</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">NULL</span><span class="p">,</span><span class="w"> </span><span class="n">lat</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">NULL</span><span class="p">,</span><span class="w"> </span><span class="n">lon</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">NULL</span><span class="p">,</span><span class="w"> </span><span class="n">type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">NULL</span><span class="p">,</span><span class="w"> </span><span class="n">postcode</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">NULL</span><span class="p">,</span><span class="w"> </span><span class="n">citycode</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">NULL</span><span class="p">){</span><span class="w">
  </span><span class="n">args</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="n">q</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">q</span><span class="p">,</span><span class="w"> </span><span class="n">limit</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">limit</span><span class="p">,</span><span class="w"> </span><span class="n">autocomplete</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">autocomplete</span><span class="p">,</span><span class="w"> </span><span class="n">lat</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">lat</span><span class="p">,</span><span class="w"> 
               </span><span class="n">lon</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">lon</span><span class="p">,</span><span class="w"> </span><span class="n">type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">type</span><span class="p">,</span><span class="w"> </span><span class="n">postcode</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">postcode</span><span class="p">,</span><span class="w"> </span><span class="n">citycode</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">citycode</span><span class="p">)</span><span class="w">
  </span><span class="c1"># Check that at least one argument is not null</span><span class="w">
  </span><span class="n">stop_if_all</span><span class="p">(</span><span class="n">args</span><span class="p">,</span><span class="w"> </span><span class="n">is.null</span><span class="p">,</span><span class="w"> </span><span class="s2">"You need to specify at least one argument"</span><span class="p">)</span><span class="w">
  </span><span class="c1"># Chek for internet</span><span class="w">
  </span><span class="n">check_internet</span><span class="p">()</span><span class="w">
  </span><span class="c1"># Create the </span><span class="w">
  </span><span class="n">res</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">GET</span><span class="p">(</span><span class="n">base_url</span><span class="p">,</span><span class="w"> </span><span class="n">query</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">compact</span><span class="p">(</span><span class="n">args</span><span class="p">))</span><span class="w">
  </span><span class="c1"># Check the result</span><span class="w">
  </span><span class="n">check_status</span><span class="p">(</span><span class="n">res</span><span class="p">)</span><span class="w">
  </span><span class="c1"># Get the content and return it as a data.frame</span><span class="w">
  </span><span class="n">fromJSON</span><span class="p">(</span><span class="n">rawToChar</span><span class="p">(</span><span class="n">res</span><span class="o">$</span><span class="n">content</span><span class="p">))</span><span class="o">$</span><span class="n">features</span><span class="w">
</span><span class="p">}</span><span class="w">

</span><span class="cd">#' @export</span><span class="w">
</span><span class="cd">#' @rdname searchban</span><span class="w">
</span><span class="n">reverse_search_ban</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="k">function</span><span class="p">(</span><span class="n">lat</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">NULL</span><span class="p">,</span><span class="w"> </span><span class="n">lon</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">NULL</span><span class="p">){</span><span class="w">
  </span><span class="n">args</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="n">lat</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">lat</span><span class="p">,</span><span class="w"> </span><span class="n">lon</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">lon</span><span class="p">)</span><span class="w">
  </span><span class="c1"># Check that at least one argument is not null</span><span class="w">
  </span><span class="n">stop_if_all</span><span class="p">(</span><span class="n">args</span><span class="p">,</span><span class="w"> </span><span class="n">is.null</span><span class="p">,</span><span class="w"> </span><span class="s2">"You need to specify at least one argument"</span><span class="p">)</span><span class="w">
  </span><span class="c1"># Chek for internet</span><span class="w">
  </span><span class="n">check_internet</span><span class="p">()</span><span class="w">
  </span><span class="c1"># Create the </span><span class="w">
  </span><span class="n">res</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">GET</span><span class="p">(</span><span class="n">reverse_url</span><span class="p">,</span><span class="w"> </span><span class="n">query</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">compact</span><span class="p">(</span><span class="n">args</span><span class="p">))</span><span class="w">
  </span><span class="c1"># Check the result</span><span class="w">
  </span><span class="n">check_status</span><span class="p">(</span><span class="n">res</span><span class="p">)</span><span class="w">
  </span><span class="c1"># Get the content and return it as a data.frame</span><span class="w">
  </span><span class="n">fromJSON</span><span class="p">(</span><span class="n">rawToChar</span><span class="p">(</span><span class="n">res</span><span class="o">$</span><span class="n">content</span><span class="p">))</span><span class="o">$</span><span class="n">features</span><span class="w">
</span><span class="p">}</span><span class="w">
</span>

Note: you’ll need to change the arguments and documentation for your
specific API (obviously).

If ever you want to a part of this roxygen filling programmatically, you
should check the excellent {sinew}
package by Jonathan Sidi.

Step 6 : Roxygenise

Now, run in your console :

<span class="n">roxygen2</span><span class="o">::</span><span class="n">roxygenise</span><span class="p">()</span><span class="w">
</span>

And that’s it! You’ve got a working package 🙂

Step 7 : build your package

You can test that everything is ok with:

<span class="n">devtools</span><span class="o">::</span><span class="n">check</span><span class="p">()</span><span class="w">
</span><span class="c1"># Then build it with:</span><span class="w">
</span><span class="n">devtools</span><span class="o">::</span><span class="n">build</span><span class="p">()</span><span class="w">
</span>

What’s left

As I said, the 10 minutes workflow does not include the doc, so here’s
what left for you to do:

Write the Readme
Write a Vignette
Write tests

For that, go back to your dev file, add, and run:

<span class="n">use_testthat</span><span class="p">()</span><span class="w">
</span><span class="n">use_vignette</span><span class="p">(</span><span class="s2">"{your-package-name}"</span><span class="p">)</span><span class="w">
</span><span class="n">use_readme_rmd</span><span class="p">()</span><span class="w">
</span>

And now, grab you best pen and write documentation 😉

To leave a comment for the author, please follow the link and comment on their blog: Colin Fay.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.

Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

R-bloggers

R news and tutorials contributed by hundreds of R bloggers

[How to] Build an API wrapper package in 10 minutes.

Find the API

Step 0: be sure you have all the packages you’ll need

Step 1: the project

Step 2: devstuffs

Step 3: get the API url

Step 4: utils

Step 5 : the function that will call on the API

Step 6 : Roxygenise

Step 7 : build your package

What’s left

Related

Find the API

Step 0: be sure you have all the packages you’ll need

Step 1: the project

Step 2: devstuffs

Step 3: get the API url

Step 4: utils

Step 5 : the function that will call on the API

Step 6 : Roxygenise

Step 7 : build your package

What’s left

Related

Never miss an update! Subscribe to R-bloggers to receive e-mails with the latest R posts. (You will not see this message again.)

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)