I’m very happy to announce that
reqres has been released on CRAN.
a new (in R context) approach to working with HTTP messages, that is, the
requests you send to a server and the respons it returns. The
uunderlying mechanics of a web server is seldom something that R users comes
into contact with, indeed the most popular way of using R code for the web is
Shiny by RStudio and
OpenCPU by Jeroen Ooms, both of which abstracts the
actual HTTP messaging away in order to provide a more friendly and R-native
interface to building web apps and services.
So why bother with HTTP in R at all?
Both of the above frameworks favors ease-of-use over control, and sometimes you just want control. Maybe you don’t need the overhead that comes with full-fledged web app frameworks, maybe the high abstraction level makes it difficult to achieve what you want, or maybe you’re the developer of Shiny or OpenCPU and wants to declutter your codebase ?. I’m of course here to tell you to take a look at
(which I’m developing) if you can give a nod to the two former reasons. I’m not
going to spend more time on how and why to build a web server (that will happen
in another series of blog posts), but simply state that there are very valid
reasons for working directly with HTTP messaging in R and that
reqres is here
to soothe the pain of it, should you ever be in that position.
An overview of reqres
There are two main objects in
reqres that the developer should know about. The
Request class and the
Response class. Both of these are build on
R6 and heavily inspired by the request and
response classes in Express.js (a web server framework
for Node.js) to the point of seeming very familiar if
you’ve ever worked with Express.
The Request class
When a HTTP request is recieved on the server the most likely way it ends up in
R is through the
httpuv package, a
minimal web server build upon libuv (which were developed for Node – we’re
httpuv passes the request on as
a Rook compliant environment (Rook
being an earlier web server specification developed by Jeffrey Horner) and this
is the point where
reqres can intercept it and make your life easier.
In order to showcase the
Request class we need a HTTP request. Thankfully
fiery provides a function for mocking Rook requests, so we can play around
reqres without building up a true server.
We now have a supercharged
Request object. Being an
R6 class it uses
reference semantics and there will thus only exist one version of this request
no matter how many times we reassign it to a new variable. We can ask the
request all sorts on things about itself, such as the method, full url, path
(the part of the url following the host address), the query (the optional part
of the url following the
As can be seen the query gets parsed automatically into a list. The same is true for cookies. One surprise might come when we try to look at the body.
The body is only parsed on request. The reason for this is that the request body
can be in all sorts of formats, some not even understandable by R. The format of
the body is advertised in the
Content-Type header (here
and we can ask the request whether it is of a certain format.
This can be used to determine the correct approach to reading the body. An
easier way is through the
parser() method that takes multiple different
parsing functions and chooses the correct (if present) and fills in the body.
reqres already comes with a list of parsers for the standard formats so often
this is a very easy task.
The method returns
TRUE if successful and
FALSE otherwise. One little magic
feature is that the body is automatically decompressed if compressed (e.g.
Arbitrary headers can be extracted with the
The last major feature of the
Request class is content negotiation. It is
expected that the client informs the server what format, encoding, etc it
understands and prefers and the server then choses the best one it can do (this
is communicated through the
Accept(-*) headers). The
Request class has a
range of methods that helps you chose the correct format of the response body.
While the content negotiation seems relatively simple in our little contrived
example, it can easily end up being hairy as each format can be weighted by a
priority score and wildcards should be prioritized less. The
takes care of all of this for you and simply returns the prefered choice out of
The Response class
Request class is mainly meant for parsing and reading the intend of
Response class is meant for manipulation, ultimately resulting in
an answer to the
Request. A response is always linked to a request and cannot
exist in solitude. While it can be created using the standard
Response$new() ideom, it is recommended to create one from the request
The reason why this is recommended is that the
respond() method will always
return a response, either creating one or returning the one already exisiting.
Response$new() will throw an error if a response has already been created for
Responses are initialised to
404- Not Found with an empty body but it is often
desirable to change this (unless, of course, the requested ressource is not
found). Status codes can be manipulated with the
status field or the
status_with_text() method which will also update the body to contain the name
of the status code, e.g.
Headers can be set with the
append_header() method and
set_header() method is the lowest common denominator when it comes to
adding headers to your response. In addition there are a range of helpers for
specific common headers.
Furtermore, there’s a special method for setting cookies. While cookies are set
Set-Cookie header, they live in a separate container until the
response is ready to be send in order to facilitate lookup by cookie name.
Apart from the headers, the each response also contains their own data store that can contain any data. This facilitate communication between different middleware (code that modify the HTTP messages on the server). The data store is used pretty much like the headers.
The data contained in the data store will never become part of the actual response so anything can be added here safely.
Often the server responds with a file, e.g. a HTML file defining the web page
the client requested. Files are easily added with the
file field, which will
take care of setting the
Last-Modified headers as well as
checking that the file actually exists.
If the file is meant for download rather than display the
attach() method will
set the correct headers to indicate to the browser that it should initiate a
Lastly, the response body is accessible in the
body field. It can be
absolutely anything you wish as until the response is send of, but should be
formatted to either a raw vector or a string prior to handing the response of to
httpuv. Thankfully there’s a parallel to
Request$parse() in the form of
Response$format that performs content negotiation based on the supplied
formatters, chooses the prefered one and applies it, finally applying
compression if the client permits it. To make life easier the standard
formatters have been collected in
default_formatters so this step is
easy-peasy (headers will of course be set for you).
As I hope I’ve made clear, working directly with HTTP messages does not need to
be a drag. Sure, there’s a sea of conventions around headers and status codes
that can seem daunting, but
reqres takes care of the minimum requirements for
you, letting you focus on the server logic instead.
I obviously hope
reqres will become pervasive in the world of R web
technologies as I think it will make everyones life easier.
fiery already uses
it in the development version on GitHub, as does
routr so if you’re going to
use either of these packages you’ll automatically become aquainted. Furthermore
I’ve heard expression of interest from other developers so hopefully it will be
adopted beyond my own packages.