[How to] Build an API wrapper package in 10 minutes.

January 20, 2018
By

(This article was first published on Colin Fay, and kindly contributed to R-bloggers)

Well… documentation not included (of course).

API are cool. They allow to retrieve data from the web, and if ever
you’re familiar with {httr}, {jsonlite} and packages like these,
you’re able to build requests and retrieve data in a matter of
minutes.

But no worry, if you’re not familiar with http requests and web formats
like html, JSON and such, you can still go and look for a package
wrapper around that specific API: someone had probably been coding it
already.

And, if you want to be that someone that code an API wrapper package,
here’s a short tutorial that will allow you to create it.

Disclaimer: the 10 minutes workflow does not (of course) include the
package documentation.

Find the API

Well, this first step can take a while… but, let’s say we have already
found it, and want to build an package around the french database for
addresses: https://adresse.data.gouv.fr/api/.

Step 0: be sure you have all the packages you’ll need

Run this if you want to be sure you have all the packages we’ll use
here:

install.packages("devtools")
install.packages("roxygen2")
install.packages("usethis")
install.packages("curl")
install.packages("httr")
install.packages("jsonlite")
install.packages("attempt")
install.packages("purrr")
devtools::install_github("r-lib/desc")

Step 1: the project

Create a new project with RStudio, and click on “Create a package with
devtools”.

Step 2: devstuffs

  • In your terminal, run usethis::use_data_raw().

  • Open a new R Script, save it into /data_raw as “devstuffs”, or any
    other name. Copy and paste into this script:

library(devtools)
library(usethis)
library(desc)

# Remove default DESC
unlink("DESCRIPTION")
# Create and clean desc
my_desc <- description$new("!new")

# Set your package name
my_desc$set("Package", "yourpackage")

#Set your name
my_desc$set("[email protected]", "person('Colin', 'Fay', email = '[email protected]', role = c('cre', 'aut'))")

# Remove some author fields
my_desc$del("Maintainer")

# Set the version
my_desc$set_version("0.0.0.9000")

# The title of your package
my_desc$set(Title = "My Supper API Wrapper")
# The description of your package
my_desc$set(Description = "A long description of this super package I'm working on.")
# The urls
my_desc$set("URL", "http://this")
my_desc$set("BugReports", "http://that")
# Save everyting
my_desc$write(file = "DESCRIPTION")

# If you want to use the MIT licence, code of conduct, and lifecycle badge
use_mit_license(name = "Colin FAY")
use_code_of_conduct()
use_lifecycle_badge("Experimental")
use_news_md()

# Get the dependencies
use_package("httr")
use_package("jsonlite")
use_package("curl")
use_package("attempt")
use_package("purrr")

# Clean your description
use_tidy_description()

Then, run everything from this script.

Copy and paste at the top of you README.Rmd
:

  [![lifecycle](https://img.shields.io/badge/lifecycle-experimental-orange.svg)](https://www.tidyverse.org/lifecycle/#experimental)

And at the bottom
:

  Please note that this project is released with a [Contributor Code of Conduct](CODE_OF_CONDUCT.md).
  By participating in this project you agree to abide by its terms.

Step 3: get the API url

There are two ways you can request on an API :

  • By building url: url.and/path/to/the/data

  • With parameters: url.that/q=this&data=that

This is the case for https://adresse.data.gouv.fr/api/. We’ve got the
base url: http 'https://api-adresse.data.gouv.fr/search/?q=8 bd du
port'
, and search parameters.

Here, the base url is everything before the ?, and the parameters are
the key-value pairs after the
.

Step 4: utils

Before creating the main calling functions, we’ll create some utilitary
functions that will run some tests: is the internet connexion running?
Does the {httr} result return the right http code?

For this, we’ll create a file called utils.R, save it in the R/
folder, and put into it:

#' @importFrom attempt stop_if_not
#' @importFrom curl has_internet
check_internet <- function(){
  stop_if_not(.x = has_internet(), msg = "Please check your internet connexion")
}

#' @importFrom httr status_code
check_status <- function(res){
  stop_if_not(.x = status_code(res), 
              .p = ~ .x == 200,
              msg = "The API returned an error")
}

In this same file we’ll also create two objects that will contain the
base API urls:

base_url <- "https://api-adresse.data.gouv.fr/search/"
reverse_url <- "https://api-adresse.data.gouv.fr/reverse/"

Step 5 : the function that will call on the API

To call the API, we’ll use GET function from {httr}. Let’s first try
just this:

httr::GET(url = base_url, query = list(q = "Yeaye"))
## Response [https://api-adresse.data.gouv.fr/search/?q=Yeaye]
##   Date: 2018-02-04 21:40
##   Status: 200
##   Content-Type: application/json; charset=utf-8
##   Size: 574 B

As you can see, the status is 200. Which is a good sign: no error from
the API.

In the API we have chosen, there are 4 entry points: search, reverse,
and their counterparts with csv. These counterparts work by POSTing a
csv to the API, so let’s forget them for now.

The search entrypoint can take 8 parameters:

  • q: text search
  • limit: maximum number of results to return
  • autocomplete: autocompletion
  • lat: latitude
  • lon: longitude
  • type: type of elements to return (housenumber, street,place,
    municipality)
  • postcode: Postal code
  • citycode: INSEE code

These are the elements which will be passed as a list in the query
parameter from httr::GET.

Note: if you’re going for an url based request
(url.and/path/to/the/data or url.and/?path=this&to=that), you
don’t need to set the query parameter, you can simply paste the url
with the elements (url <- paste0("url.and/?path=", this, "&to=",
that)
).

Create a new R script and write the functions used to call the API:

#' Search the BAN
#' 
#' @param q text search
#' @param limit maximum number of results to return 
#' @param autocomplete autocompletion
#' @param lat latitude
#' @param lon longitude
#' @param type type of elements to return (housenumber, street,place, municipality)
#' @param postcode Postal code
#' @param citycode INSEE code
#'
#' @importFrom attempt stop_if_all
#' @importFrom purrr compact
#' @importFrom jsonlite fromJSON
#' @importFrom httr GET
#' @export
#' @rdname searchban
#'
#' @return the results from the search
#' @examples 
#' \dontrun{
#' search_ban("Rennes")
#' reverse_search_ban("48.11", "-1.68")
#' }

search_ban <- function(q = NULL, limit = NULL, autocomplete = NULL, lat = NULL, lon = NULL, type = NULL, postcode = NULL, citycode = NULL){
  args <- list(q = q, limit = limit, autocomplete = autocomplete, lat = lat, 
               lon = lon, type = type, postcode = postcode, citycode = citycode)
  # Check that at least one argument is not null
  stop_if_all(args, is.null, "You need to specify at least one argument")
  # Chek for internet
  check_internet()
  # Create the 
  res <- GET(base_url, query = compact(args))
  # Check the result
  check_status(res)
  # Get the content and return it as a data.frame
  fromJSON(rawToChar(res$content))$features
}

#' @export
#' @rdname searchban
reverse_search_ban <- function(lat = NULL, lon = NULL){
  args <- list(lat = lat, lon = lon)
  # Check that at least one argument is not null
  stop_if_all(args, is.null, "You need to specify at least one argument")
  # Chek for internet
  check_internet()
  # Create the 
  res <- GET(reverse_url, query = compact(args))
  # Check the result
  check_status(res)
  # Get the content and return it as a data.frame
  fromJSON(rawToChar(res$content))$features
}

Note: you’ll need to change the arguments and documentation for your
specific API (obviously).

If ever you want to a part of this roxygen filling programmatically, you
should check the excellent {sinew}
package
by Jonathan Sidi.

Step 6 : Roxygenise

Now, run in your console :

roxygen2::roxygenise()

And that’s it! You’ve got a working package 🙂

Step 7 : build your package

You can test that everything is ok with:

devtools::check()
# Then build it with:
devtools::build()

What’s left

As I said, the 10 minutes workflow does not include the doc, so here’s
what left for you to do:

  • Write the Readme
  • Write a Vignette
  • Write tests

For that, go back to your dev file, add, and run:

use_testthat()
use_vignette("{your-package-name}")
use_readme_rmd()

And now, grab you best pen and write documentation 😉

To leave a comment for the author, please follow the link and comment on their blog: Colin Fay.

R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...



If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Comments are closed.

Search R-bloggers

Sponsors

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)