Getting your web application and R(Apache) to talk to each other

April 19, 2010
By

(This article was first published on What You're Doing Is Rather Desperate » R, and kindly contributed to R-bloggers)

Here’s the situation. Web applications, built using a framework (e.g. Rails, Django) are great for fetching data from a database and rendering it. They’re not so great for crunching and charting the data. Conversely, R is great for crunching and charting, but doesn’t make for a great web application.

rapache-rails

Index view for values


The idea then, is to let each do what it does best and enable the passing of data between them. There isn’t a whole lot of literature on this topic, but there are a couple of guides:

  • In these seminar slides (PDF), Jeroen Ooms describes how data can be passed between a web browser and R. Briefly, he uses client-side javascript to format the data as a JSON string. Server-side R (RApache) then parses the POST variable using fromJSON() (in the rjson package), formats the results of an R function as JSON using toJSON() and sends them back to the browser.
  • Slide 46 of this presentation by Mike Driscoll of Dataspora illustrates a different approach, where a Django-based web application sends data to RApache in CSV format.

As a first step in understanding all of this, we can build a small demo application using Rails (version 2.3.5), which serves both JSON and CSV. We’ll see if we can get that into R, then see if R can return results back to Rails, via RApache. Baby steps, so we’ll avoid the AJAX stuff for now and just use Rails rendering methods to serve JSON from a controller.

1. Generate the Rails application
First, generate the skeleton. I’m calling the application “rapache” and scaffolding for a simple model, named Value, containing 2 integer fields named “x” and “y”:

rails rapache
cd rapache
rake db:create
./script/generate scaffold Value x:integer y:integer
rake db:migrate

Next, edit config/environment.rb to contain these lines, required for Rails to render CSV:

config.gem "fastercsv"
config.gem "comma"

Install those gems if you don’t have them and optionally, unpack them into your application tree. I like to freeze the Rails gems too:

rake gems:install
rake gems:unpack:dependencies
rake rails:freeze:gems

We also need to edit models/values.rb to provide the comma method:

class Value < ActiveRecord::Base
  comma do |f|
    f.x
    f.y
  end
end

So far, so good. Now, we could seed the database with some test data but for just a few values, it’s as easy to use the forms generated by the scaffolding and enter some X and Y values. I made my Y = X, fired up ./script/server and then opened up “http://localhost:3000/values” in the browser, where I saw a view like that in the image, above-right in this blog post.

Generating the JSON and CSV end-points couldn’t be easier. Just edit the index method in controllers/values_controller.rb to look like this:

def index
  @values = Value.all
  respond_to do |format|
    format.html # index.html.erb
    format.xml  { render :xml => @values }
    format.json { render :json => @values }
    format.csv  { render :csv => @values }
  end
end

That gives us JSON in the browser when we go to “http://localhost:3000/values.json” and a CSV file when we go to “http://localhost:3000/values.csv”. Depending on your browser setup, the browser may download the CSV, offer to download it or offer to open it in a spreadsheet application.

2. Experiments in R
With the Rails development environment still running, I opened an R console. First, fetching the data from the Rails application via JSON:

library(rjson)
data.json <- fromJSON(readLines(url("http://localhost:3000/values.json")))
# Warning message:
# In readLines(url("http://localhost:3000/values.json")) :
# incomplete final line found on 'http://localhost:3000/values.json'
length(data.json)
[1] 6
data.json[[1]]$value$x
[1] 0
data.json[[1]]$value$y
[1] 0

Not too bad. The warning message arises because readLines() can’t detect an end of line character; the message could be hidden using suppressWarnings(). We could use either rjson or RJSONIO – both of them have the fromJSON() method, but neither can read directly from a URL connection, hence the requirement for readLines(url(…)).
R read the JSON into a list of lists. That could be transformed into a dataframe, but it’s a little unwieldy. Let’s have a look at the CSV approach:

data.csv <- read.csv("http://localhost:3000/values.csv", header = T)
dim(data.csv)
# [1] 6 2
data.csv
#   X Y
# 1 0 0
# 2 1 1
# 3 2 2
# 4 3 3
# 5 4 4
# 6 5 5

Perfect – read.csv() can read the Rails-rendered CSV directly into a data frame.

3. Calling R from within Rails and displaying the results
The last step is to call R and retrieve results from within the Rails application. For this, you’ll need to have installed RApache. That’s beyond the scope of this post – it’s not difficult, under Ubuntu at least. I assume that R scripts will be served from /var/www/R, which is configured using the Apache configuration file /etc/apache2/conf.d/rapache.conf:

<Location "/R">
  ROutputErrors
  SetHandler r-script
  RHandler sys.source
  REvalOnStartup "library(lattice)"  # not used here
</Location>

What we should really do is write a Rails controller method to retrieve the appropriate data and then write a generic R function to accept CSV from any URL and return the results. However, for demonstration purposes, I’m just going to write a test R script, /var/www/R/plot.R, to plot a scatterplot of the X versus Y data and then wrap the result inside a Rails image_tag. Here’s the R:

setContentType("image/png")
d <- read.csv("http://localhost:3000/values.csv", header = T)
t <- tempfile()
png(t,type="cairo")
plot(d, main = "Y = X!")
dev.off()
sendBin(readBin(t,'raw',n=file.info(t)$size))
unlink(t)
DONE

And here’s the Rails – I just added this to the bottom of views/values/index.html.erb:

<div style="position:absolute; top:50px; left:400px;">
  <%= image_tag "http://localhost/R/plot.R" %>
</div>

The result – see image, right.

rapache-rails-plot

Index view for values + plot from RApache


That covers the basics of serving JSON or CSV from Rails to RApache and sending a result back to Rails. Obviously, a real application would require more model or controller methods, views, R functions, error checks and prettier views, perhaps with a dash of AJAX thrown in. You’d probably also store your R scripts in the Rails directory tree and serve them from there. On the whole though, I’d say it’s surprisingly easy to create a RESTful API using Rails and use it to communicate with R.


Filed under: programming, R, rails, research diary, ruby, statistics

To leave a comment for the author, please follow the link and comment on his blog: What You're Doing Is Rather Desperate » R.

R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...



If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Tags: , , , , ,

Comments are closed.