Computers break down every now and again and this is a serious problem when an application needs runs on thousands of individual computers (nodes) plugged together; lots more hardware creates lots more opportunity for a failure that renders any subsequent calculations by working nodes possible wrong. The solution is checkpointing; ... [Read more...]

This is probably the coolest-looking thing I’ve figured out how to do with raster images in R. Similar to (although not quite as impressive as) these images by Jeff Clark, I alter the simple k-means approach described in the previous post to... [Read more...]

A common scenario that I run into is time and how to deal with it. I often will do a  variety of summaries and analysis that need to be measured at different points in time. Whether I want to graph the data or review the results I need to be ... [Read more...]

Having spent a chink of the weekend and a piece of yesterday trying to pull NHS Winter sitrep data into some sort of shape in Scraperwiki, (described, in part, here: When Machine Readable Data Still Causes “Issues” – Wrangling Dates…), I couldn’t but help myself last night and had a ... [Read more...]

Since we are in the “big data” Era, which means that a massive amount of data are made daily available by governments, institutions, and ordinary people like you and me—the importance of simulate data for “frequentists”, like myself, seems to fade. After all, once we have access to “real ... [Read more...]

Most statisticians know that pie charts are a terrible way to plot percentages. You can find explanations here, here, and here as well as the R help file for the pie function which states: Pie charts are a very bad … Continue reading → [Read more...]

Ajay Ohri, author of R for Business Analytics, was recently interviewed about using R instead of Excel for human resources management. Human Resources is an increasingly analytics-driven field, with predictive modeling now used to prioritize applicants for interview and to organize workspaces, to give just two examples. As Ajay points ... [Read more...]

A the new Rcpp release 0.10.1 arrived this morning on CRAN (as already has Windows binaries) and in Debian. This is a follow-up to the recent 0.10.0 release which extends the exciting new Rcpp-attributes and Rcpp-sugar work further, and as in a n... [Read more...]

Richard Everitt tweetted yesterday about a recent publication in JCGS by Rajib Paul, Steve MacEachern and Mark Berliner on convergence assessment via stratification. (The paper is free-access.) Since this is another clear interest of mine’s, I had a look at the paper in the train to Besançon. (And ... [Read more...]

The other day I stumbled accross an amazing PMML model API called jpmml.  It's written in Java and supports PMML 4.1 (and older).  Neural networks, random forests, regression and trees PMML models can be consumed and used for scoring.I decide... [Read more...]

This article lists the steps I take to create a network animation in R, provides some example source code that you can copy and modify for your own work, and starts a discussion about programming and visualization as an interpretive approach in research. Before I start, take a look at ... [Read more...]

For some time I have been stealing an hour here and there for the rrdf package for R. This package is based on Apache Jena and allows reading and writing of RDF triples, as well as doing local and remote SPARQL querying. BTW, rrdf is not only R pa... [Read more...]

Just a fast note: I came across the R-package "knitr" which enables you to generate PDF files by mixing LaTeX and R code in one document. The result looks very nice and is great to create documentations, manuals and so on. I find knitr much easier to u... [Read more...]

The idea of deriving color palettes from images has long been an interest of mine. There are places you can do this online, but it’s at least plausible that you might want to identify dominant colors programatically, for a large number of image... [Read more...]

Users of the world’s favourite Bayesian analysis software WinBUGS will recognise the strange format in which data is supplied inside the GUI, generally in a text file looking something like this: list(x=c(1,2,3,4,5,6),y=c(4,8,2,6,9,9)) This is of course an R (or strictly … Continue reading → [Read more...]

I really should make it a habit of using data.table. The speed and simplicity of this R package are astonishing. Here is a simple example: I have a data frame showing incremental claims development by line of business and origin year. Now I would like add a column with ... [Read more...]

The selectr package translates a CSS selector into an equivalent XPath expression. This allows the use of CSS selectors to query XML documents using the XML package. Convenience functions are also provided to mimic functionality present in modern web browsers. … Continue reading → [Read more...]

In 1998, John Chambers won the ACM Software Systems award for the S language, which the ACM heralded as having "forever altered how people analyze, visualize, and manipulate data". The R language is "not unlike S", and John continues pushing the frontiers of statistical computing as a member of the R ... [Read more...]

So, you know how to load PNG files, access their innards, and plot them as points in a scatter plot. But, is it possible to use R to do some of the awesome things you can do with Mathematica? Well, in this series of posts, I’ll introduce just ... [Read more...]

Celebrating that I am contributing to the R-bloggers.com blog aggregator I am going to post a very simple function to check which species (both plants and animals) are considered “invaders” somewhere in the world. Basically the function asks that to … Continue reading → [Read more...]