[How to] Build an API wrapper package in 10 minutes.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.
Well… documentation not included (of course).
API are cool. They allow to retrieve data from the web, and if ever
you’re familiar with {httr}, {jsonlite} and packages like these,
you’re able to build requests and retrieve data in a matter of
minutes.
But no worry, if you’re not familiar with http requests and web formats
like html, JSON and such, you can still go and look for a package
wrapper around that specific API: someone had probably been coding it
already.
And, if you want to be that someone that code an API wrapper package,
here’s a short tutorial that will allow you to create it.
Disclaimer: the 10 minutes workflow does not (of course) include the
package documentation.
Find the API
Well, this first step can take a while… but, let’s say we have already
found it, and want to build an package around the french database for
addresses: https://adresse.data.gouv.fr/api/.
Step 0: be sure you have all the packages you’ll need
Run this if you want to be sure you have all the packages we’ll use
here:
<span class="n">install.packages</span><span class="p">(</span><span class="s2">"devtools"</span><span class="p">)</span><span class="w">
</span><span class="n">install.packages</span><span class="p">(</span><span class="s2">"roxygen2"</span><span class="p">)</span><span class="w">
</span><span class="n">install.packages</span><span class="p">(</span><span class="s2">"usethis"</span><span class="p">)</span><span class="w">
</span><span class="n">install.packages</span><span class="p">(</span><span class="s2">"curl"</span><span class="p">)</span><span class="w">
</span><span class="n">install.packages</span><span class="p">(</span><span class="s2">"httr"</span><span class="p">)</span><span class="w">
</span><span class="n">install.packages</span><span class="p">(</span><span class="s2">"jsonlite"</span><span class="p">)</span><span class="w">
</span><span class="n">install.packages</span><span class="p">(</span><span class="s2">"attempt"</span><span class="p">)</span><span class="w">
</span><span class="n">install.packages</span><span class="p">(</span><span class="s2">"purrr"</span><span class="p">)</span><span class="w">
</span><span class="n">devtools</span><span class="o">::</span><span class="n">install_github</span><span class="p">(</span><span class="s2">"r-lib/desc"</span><span class="p">)</span><span class="w">
</span>
Step 1: the project
Create a new project with RStudio, and click on “Create a package with
devtools”.
Step 2: devstuffs
-
In your terminal, run
usethis::use_data_raw()
. -
Open a new R Script, save it into
/data_raw
as “devstuffs”, or any
other name. Copy and paste into this script:
<span class="n">library</span><span class="p">(</span><span class="n">devtools</span><span class="p">)</span><span class="w">
</span><span class="n">library</span><span class="p">(</span><span class="n">usethis</span><span class="p">)</span><span class="w">
</span><span class="n">library</span><span class="p">(</span><span class="n">desc</span><span class="p">)</span><span class="w">
</span><span class="c1"># Remove default DESC</span><span class="w">
</span><span class="n">unlink</span><span class="p">(</span><span class="s2">"DESCRIPTION"</span><span class="p">)</span><span class="w">
</span><span class="c1"># Create and clean desc</span><span class="w">
</span><span class="n">my_desc</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">description</span><span class="o">$</span><span class="n">new</span><span class="p">(</span><span class="s2">"!new"</span><span class="p">)</span><span class="w">
</span><span class="c1"># Set your package name</span><span class="w">
</span><span class="n">my_desc</span><span class="o">$</span><span class="n">set</span><span class="p">(</span><span class="s2">"Package"</span><span class="p">,</span><span class="w"> </span><span class="s2">"yourpackage"</span><span class="p">)</span><span class="w">
</span><span class="c1">#Set your name</span><span class="w">
</span><span class="n">my_desc</span><span class="o">$</span><span class="n">set</span><span class="p">(</span><span class="s2">"[email protected]"</span><span class="p">,</span><span class="w"> </span><span class="s2">"person('Colin', 'Fay', email = '[email protected]', role = c('cre', 'aut'))"</span><span class="p">)</span><span class="w">
</span><span class="c1"># Remove some author fields</span><span class="w">
</span><span class="n">my_desc</span><span class="o">$</span><span class="n">del</span><span class="p">(</span><span class="s2">"Maintainer"</span><span class="p">)</span><span class="w">
</span><span class="c1"># Set the version</span><span class="w">
</span><span class="n">my_desc</span><span class="o">$</span><span class="n">set_version</span><span class="p">(</span><span class="s2">"0.0.0.9000"</span><span class="p">)</span><span class="w">
</span><span class="c1"># The title of your package</span><span class="w">
</span><span class="n">my_desc</span><span class="o">$</span><span class="n">set</span><span class="p">(</span><span class="n">Title</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"My Supper API Wrapper"</span><span class="p">)</span><span class="w">
</span><span class="c1"># The description of your package</span><span class="w">
</span><span class="n">my_desc</span><span class="o">$</span><span class="n">set</span><span class="p">(</span><span class="n">Description</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"A long description of this super package I'm working on."</span><span class="p">)</span><span class="w">
</span><span class="c1"># The urls</span><span class="w">
</span><span class="n">my_desc</span><span class="o">$</span><span class="n">set</span><span class="p">(</span><span class="s2">"URL"</span><span class="p">,</span><span class="w"> </span><span class="s2">"http://this"</span><span class="p">)</span><span class="w">
</span><span class="n">my_desc</span><span class="o">$</span><span class="n">set</span><span class="p">(</span><span class="s2">"BugReports"</span><span class="p">,</span><span class="w"> </span><span class="s2">"http://that"</span><span class="p">)</span><span class="w">
</span><span class="c1"># Save everyting</span><span class="w">
</span><span class="n">my_desc</span><span class="o">$</span><span class="n">write</span><span class="p">(</span><span class="n">file</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"DESCRIPTION"</span><span class="p">)</span><span class="w">
</span><span class="c1"># If you want to use the MIT licence, code of conduct, and lifecycle badge</span><span class="w">
</span><span class="n">use_mit_license</span><span class="p">(</span><span class="n">name</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"Colin FAY"</span><span class="p">)</span><span class="w">
</span><span class="n">use_code_of_conduct</span><span class="p">()</span><span class="w">
</span><span class="n">use_lifecycle_badge</span><span class="p">(</span><span class="s2">"Experimental"</span><span class="p">)</span><span class="w">
</span><span class="n">use_news_md</span><span class="p">()</span><span class="w">
</span><span class="c1"># Get the dependencies</span><span class="w">
</span><span class="n">use_package</span><span class="p">(</span><span class="s2">"httr"</span><span class="p">)</span><span class="w">
</span><span class="n">use_package</span><span class="p">(</span><span class="s2">"jsonlite"</span><span class="p">)</span><span class="w">
</span><span class="n">use_package</span><span class="p">(</span><span class="s2">"curl"</span><span class="p">)</span><span class="w">
</span><span class="n">use_package</span><span class="p">(</span><span class="s2">"attempt"</span><span class="p">)</span><span class="w">
</span><span class="n">use_package</span><span class="p">(</span><span class="s2">"purrr"</span><span class="p">)</span><span class="w">
</span><span class="c1"># Clean your description</span><span class="w">
</span><span class="n">use_tidy_description</span><span class="p">()</span><span class="w">
</span>
Then, run everything from this script.
Copy and paste at the top of you README.Rmd
:
[](https://www.tidyverse.org/lifecycle/#experimental)
And at the bottom
:
Please note that this project is released with a [Contributor Code of Conduct](CODE_OF_CONDUCT.md).
By participating in this project you agree to abide by its terms.
Step 3: get the API url
There are two ways you can request on an API :
-
By building url:
url.and/path/to/the/data
-
With parameters:
url.that/q=this&data=that
This is the case for https://adresse.data.gouv.fr/api/. We’ve got the
base url: http 'https://api-adresse.data.gouv.fr/search/?q=8 bd du
, and search parameters.
port'
Here, the base url is everything before the ?
, and the parameters are
the key-value pairs after the
.
Step 4: utils
Before creating the main calling functions, we’ll create some utilitary
functions that will run some tests: is the internet connexion running?
Does the {httr} result return the right http code?
For this, we’ll create a file called utils.R
, save it in the R/
folder, and put into it:
<span class="cd">#' @importFrom attempt stop_if_not</span><span class="w">
</span><span class="cd">#' @importFrom curl has_internet</span><span class="w">
</span><span class="n">check_internet</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="k">function</span><span class="p">(){</span><span class="w">
</span><span class="n">stop_if_not</span><span class="p">(</span><span class="n">.x</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">has_internet</span><span class="p">(),</span><span class="w"> </span><span class="n">msg</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"Please check your internet connexion"</span><span class="p">)</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="cd">#' @importFrom httr status_code</span><span class="w">
</span><span class="n">check_status</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="k">function</span><span class="p">(</span><span class="n">res</span><span class="p">){</span><span class="w">
</span><span class="n">stop_if_not</span><span class="p">(</span><span class="n">.x</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">status_code</span><span class="p">(</span><span class="n">res</span><span class="p">),</span><span class="w">
</span><span class="n">.p</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">~</span><span class="w"> </span><span class="n">.x</span><span class="w"> </span><span class="o">==</span><span class="w"> </span><span class="m">200</span><span class="p">,</span><span class="w">
</span><span class="n">msg</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"The API returned an error"</span><span class="p">)</span><span class="w">
</span><span class="p">}</span><span class="w">
</span>
In this same file we’ll also create two objects that will contain the
base API urls:
<span class="n">base_url</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="s2">"https://api-adresse.data.gouv.fr/search/"</span><span class="w">
</span><span class="n">reverse_url</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="s2">"https://api-adresse.data.gouv.fr/reverse/"</span><span class="w">
</span>
Step 5 : the function that will call on the API
To call the API, we’ll use GET
function from {httr}. Let’s first try
just this:
<span class="n">httr</span><span class="o">::</span><span class="n">GET</span><span class="p">(</span><span class="n">url</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">base_url</span><span class="p">,</span><span class="w"> </span><span class="n">query</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="n">q</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"Yeaye"</span><span class="p">))</span><span class="w">
</span>
## Response [https://api-adresse.data.gouv.fr/search/?q=Yeaye]
## Date: 2018-02-04 21:40
## Status: 200
## Content-Type: application/json; charset=utf-8
## Size: 574 B
As you can see, the status is 200. Which is a good sign: no error from
the API.
In the API we have chosen, there are 4 entry points: search, reverse,
and their counterparts with csv. These counterparts work by POSTing a
csv to the API, so let’s forget them for now.
The search entrypoint can take 8 parameters:
q
: text searchlimit
: maximum number of results to returnautocomplete
: autocompletionlat
: latitudelon
: longitudetype
: type of elements to return (housenumber, street,place,
municipality)postcode
: Postal codecitycode
: INSEE code
These are the elements which will be passed as a list in the query
parameter from httr::GET
.
Note: if you’re going for an url based request
(url.and/path/to/the/data
orurl.and/?path=this&to=that
), you
don’t need to set the query parameter, you can simply paste the url
with the elements (url <- paste0("url.and/?path=", this, "&to=", that)
).
Create a new R script and write the functions used to call the API:
<span class="cd">#' Search the BAN</span><span class="w">
</span><span class="cd">#' </span><span class="w">
</span><span class="cd">#' @param q text search</span><span class="w">
</span><span class="cd">#' @param limit maximum number of results to return </span><span class="w">
</span><span class="cd">#' @param autocomplete autocompletion</span><span class="w">
</span><span class="cd">#' @param lat latitude</span><span class="w">
</span><span class="cd">#' @param lon longitude</span><span class="w">
</span><span class="cd">#' @param type type of elements to return (housenumber, street,place, municipality)</span><span class="w">
</span><span class="cd">#' @param postcode Postal code</span><span class="w">
</span><span class="cd">#' @param citycode INSEE code</span><span class="w">
</span><span class="cd">#'</span><span class="w">
</span><span class="cd">#' @importFrom attempt stop_if_all</span><span class="w">
</span><span class="cd">#' @importFrom purrr compact</span><span class="w">
</span><span class="cd">#' @importFrom jsonlite fromJSON</span><span class="w">
</span><span class="cd">#' @importFrom httr GET</span><span class="w">
</span><span class="cd">#' @export</span><span class="w">
</span><span class="cd">#' @rdname searchban</span><span class="w">
</span><span class="cd">#'</span><span class="w">
</span><span class="cd">#' @return the results from the search</span><span class="w">
</span><span class="cd">#' @examples </span><span class="w">
</span><span class="cd">#' \dontrun{</span><span class="w">
</span><span class="cd">#' search_ban("Rennes")</span><span class="w">
</span><span class="cd">#' reverse_search_ban("48.11", "-1.68")</span><span class="w">
</span><span class="cd">#' }</span><span class="w">
</span><span class="n">search_ban</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="k">function</span><span class="p">(</span><span class="n">q</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">NULL</span><span class="p">,</span><span class="w"> </span><span class="n">limit</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">NULL</span><span class="p">,</span><span class="w"> </span><span class="n">autocomplete</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">NULL</span><span class="p">,</span><span class="w"> </span><span class="n">lat</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">NULL</span><span class="p">,</span><span class="w"> </span><span class="n">lon</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">NULL</span><span class="p">,</span><span class="w"> </span><span class="n">type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">NULL</span><span class="p">,</span><span class="w"> </span><span class="n">postcode</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">NULL</span><span class="p">,</span><span class="w"> </span><span class="n">citycode</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">NULL</span><span class="p">){</span><span class="w">
</span><span class="n">args</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="n">q</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">q</span><span class="p">,</span><span class="w"> </span><span class="n">limit</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">limit</span><span class="p">,</span><span class="w"> </span><span class="n">autocomplete</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">autocomplete</span><span class="p">,</span><span class="w"> </span><span class="n">lat</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">lat</span><span class="p">,</span><span class="w">
</span><span class="n">lon</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">lon</span><span class="p">,</span><span class="w"> </span><span class="n">type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">type</span><span class="p">,</span><span class="w"> </span><span class="n">postcode</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">postcode</span><span class="p">,</span><span class="w"> </span><span class="n">citycode</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">citycode</span><span class="p">)</span><span class="w">
</span><span class="c1"># Check that at least one argument is not null</span><span class="w">
</span><span class="n">stop_if_all</span><span class="p">(</span><span class="n">args</span><span class="p">,</span><span class="w"> </span><span class="n">is.null</span><span class="p">,</span><span class="w"> </span><span class="s2">"You need to specify at least one argument"</span><span class="p">)</span><span class="w">
</span><span class="c1"># Chek for internet</span><span class="w">
</span><span class="n">check_internet</span><span class="p">()</span><span class="w">
</span><span class="c1"># Create the </span><span class="w">
</span><span class="n">res</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">GET</span><span class="p">(</span><span class="n">base_url</span><span class="p">,</span><span class="w"> </span><span class="n">query</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">compact</span><span class="p">(</span><span class="n">args</span><span class="p">))</span><span class="w">
</span><span class="c1"># Check the result</span><span class="w">
</span><span class="n">check_status</span><span class="p">(</span><span class="n">res</span><span class="p">)</span><span class="w">
</span><span class="c1"># Get the content and return it as a data.frame</span><span class="w">
</span><span class="n">fromJSON</span><span class="p">(</span><span class="n">rawToChar</span><span class="p">(</span><span class="n">res</span><span class="o">$</span><span class="n">content</span><span class="p">))</span><span class="o">$</span><span class="n">features</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="cd">#' @export</span><span class="w">
</span><span class="cd">#' @rdname searchban</span><span class="w">
</span><span class="n">reverse_search_ban</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="k">function</span><span class="p">(</span><span class="n">lat</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">NULL</span><span class="p">,</span><span class="w"> </span><span class="n">lon</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">NULL</span><span class="p">){</span><span class="w">
</span><span class="n">args</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="n">lat</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">lat</span><span class="p">,</span><span class="w"> </span><span class="n">lon</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">lon</span><span class="p">)</span><span class="w">
</span><span class="c1"># Check that at least one argument is not null</span><span class="w">
</span><span class="n">stop_if_all</span><span class="p">(</span><span class="n">args</span><span class="p">,</span><span class="w"> </span><span class="n">is.null</span><span class="p">,</span><span class="w"> </span><span class="s2">"You need to specify at least one argument"</span><span class="p">)</span><span class="w">
</span><span class="c1"># Chek for internet</span><span class="w">
</span><span class="n">check_internet</span><span class="p">()</span><span class="w">
</span><span class="c1"># Create the </span><span class="w">
</span><span class="n">res</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">GET</span><span class="p">(</span><span class="n">reverse_url</span><span class="p">,</span><span class="w"> </span><span class="n">query</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">compact</span><span class="p">(</span><span class="n">args</span><span class="p">))</span><span class="w">
</span><span class="c1"># Check the result</span><span class="w">
</span><span class="n">check_status</span><span class="p">(</span><span class="n">res</span><span class="p">)</span><span class="w">
</span><span class="c1"># Get the content and return it as a data.frame</span><span class="w">
</span><span class="n">fromJSON</span><span class="p">(</span><span class="n">rawToChar</span><span class="p">(</span><span class="n">res</span><span class="o">$</span><span class="n">content</span><span class="p">))</span><span class="o">$</span><span class="n">features</span><span class="w">
</span><span class="p">}</span><span class="w">
</span>
Note: you’ll need to change the arguments and documentation for your
specific API (obviously).
If ever you want to a part of this roxygen filling programmatically, you
should check the excellent {sinew}
package by Jonathan Sidi.
Step 6 : Roxygenise
Now, run in your console :
<span class="n">roxygen2</span><span class="o">::</span><span class="n">roxygenise</span><span class="p">()</span><span class="w">
</span>
And that’s it! You’ve got a working package 🙂
Step 7 : build your package
You can test that everything is ok with:
<span class="n">devtools</span><span class="o">::</span><span class="n">check</span><span class="p">()</span><span class="w">
</span><span class="c1"># Then build it with:</span><span class="w">
</span><span class="n">devtools</span><span class="o">::</span><span class="n">build</span><span class="p">()</span><span class="w">
</span>
What’s left
As I said, the 10 minutes workflow does not include the doc, so here’s
what left for you to do:
- Write the Readme
- Write a Vignette
- Write tests
For that, go back to your dev file, add, and run:
<span class="n">use_testthat</span><span class="p">()</span><span class="w">
</span><span class="n">use_vignette</span><span class="p">(</span><span class="s2">"{your-package-name}"</span><span class="p">)</span><span class="w">
</span><span class="n">use_readme_rmd</span><span class="p">()</span><span class="w">
</span>
And now, grab you best pen and write documentation 😉
R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.