Recently I played bridge with my friends. Being frustrated with several consecutive poor hand distributions we asked ourselves a question what is the probability of having a hand good enough for a small slam. A well known rule of thumb is that you need... [Read more...]

My wife went out to her book group tonight and their book of the month was 50 Shades of Grey. Sadly, I could think of is that plotting 50 shades in R would be a neat exercise. require(ggplot2) grey50 [Read more...]

Today I want to highlight the XLLoop framework : Excel User-Define Functions in in any language. The XLLoop consists of two main components: An Excel addin implementation (XLL written in c++). A server and framework written in R (or/and in many other languages). The XLLoop allows you to connect Excel ... [Read more...]

Occasionally, I will get requests from clients to calculate the mean. Most of the time it’s a simple request but from time-to-time the data was originally from grouped data. A common approach is to take the midpoint of each of the groups and just assume that all respondents within ... [Read more...]

Conrad launched the 3.6 series of Armadillo earlier today with a first 3.6.0 release. So RcppArmadillo, our wrapper for R and Armadillo, is now on CRAN with its corresponding version 0.3.6.0. No R level or interface changes were needed, and the upstr... [Read more...]

I know “officially” data scientists all always work in “big data” environments with data in a remote database, streaming store or key-value system. But in day to day work Excel files and Excel export files get used a lot and cause a disproportionate amount of pain. I would like to ... [Read more...]

[Read more...]

The popular geostatistical software SGeMS has some options for working with non-point support (block) data through the BGeost set of algorithms by Yongshe Liu (see his PhD thesis), and published in Liu and Journel (2009). A specific but ... [Read more...]

According to an analysis by Prescribing Analytics (a joint venture of technologists and doctors in the UK), Britain's cash-strapped National Health Service (NHS) is overspending on prescription drugs. While cheaper (but equally effective) generic drugs are widely available for many treatments, some doctors continue to prescribe patented drugs which can ... [Read more...]

The launch or official opening or whatever it was of the Open Data Institute this week provided another chance to grab a snapshot of notable folk in the community, as for example demonstrated by people commonly followed by users of the #ODIlaunch hashtag on Twitter. The PR campaign also resulted ... [Read more...]

Draws for the knockout phase of the 2012–13 UEFA Champions League will be held in Nyon on the 20th December 2012. The rules of the draw are simple and are as follows:8 Group winner teams will be seeded.8 Group runner-up teams will be unseeded.Teams coming from the same group and from ... [Read more...]

Today’s example is a little odd, in that the code isn’t pretty and the example isn’t really something you’d actually produce in real life — but if you’ll overlook those oddities, you’ll find that the spsample(... [Read more...]

The advent of municipal open data initiatives has been both a blessing and curse for my particular brand of data nerd. On one hand, it has opened up the possibility of developing deep and useful knowledge about the places we... [Read more...]

I’ve spent a good deal of 2012 constructing a data warehouse to manage all the various data elements that my company has. Although we’re a small enterprise, the richness and complexity of the information is rather high. Moreover, as a data-driven organization, there’s a strong impetus to construct ... [Read more...]

I've been having some great fun parallelizing R code on Amazon's cloud. Now that things are chugging away nicely, it's time to document my foibles so I can remember not to fall into the same pits of despair again. The goal was to perform lots of trails of a randomized ... [Read more...]

I have found that I get data from many different sources.  These sources range from simple .csv files to more complex relational databases, to structure XML or JSON files.  I have compiled the different approaches that one can use to easily access these datasets. Local Column Delimited Files This is ... [Read more...]

Tibshirani’s original paper on the lasso. Breiman’s Garotte — 1993 Tibshirani lasso paper submitted — 1994 Tibshirani lasso paper revised — 1995 Tibshirani lasso paper accepted — 1996 This is one of those papers that I’m so excited about, I feel like “You should just read the whole thing! It’s all good!” But I ... [Read more...]

After my post on item response theory,  a number of you have asked for links to applications that provide R code.  As I noted in that post, a good deal of work is being done in an area of research called patient-related outcome measurement (P... [Read more...]

I read the post The surprisingly weak case for global warming and the rejection; Climate: Misspecified. Based on the first, I wanted to make a post, just to write I agree with the second.The post features a number of plots like thisFor m... [Read more...]

By Revolution Analytics training manager James Peruvankal If you are new to R, and want to get an introduction to the R language, in the classic “learning by doing way”, Code school and O’Reilly have put together the Try R interactive tutorial. This tutorial is a painless introduction to ... [Read more...]