After my last post, I came across a few articles supporting the opinion that if you have a good reason to take random samples from a “big” dataset, you’re not committing some kind of sin: Big Data Blasphemy: Why Sample? … Continue reading → [Read more...]

What's with those estimates?By Ben OgorekIn R, categorical variables can be added to a regression using the lm() function without a hint of extra work. But have you ever look at the resulting estimates and wondered exactly what they were?First, let's define a data set.set.seed(12255)n = 30... [Read more...]

Pedigree based mixed models (often called animal models, due to modelling animal performance) are the cornerstone of animal breeding and quantitative genetics. There are many programs that can be used for analyzing your data with these models, e.g., ASREML, BLUPf90, MATVEC, MiXBLUP & MiX99,  SurvivalKit, PEST/VCE, WOMBAT, ...). There are ... [Read more...]

[Read more...]

[Read more...]

Update: As of 18 April 2012 the course is now fully subscribed. Should you wish to be placed on a waiting list please contact Steve Juggins. Analysing Palaeolimnological Data with R 16th — 20th August 2012, University Marine Biological Station … Continue reading → [Read more...]

The process of working on metadata and temperature series gives rise to several situations where I need to calculate the distance from every station to every other station. With a small number of stations this can be done easily on the fly with the result stored in a matrix. The ... [Read more...]

Paulo (from the Instituto de Matemática e Estatística, Universidade de São Paulo, Brazil) has posted an answer to my earlier question both as a comment on the ‘Og and as a solution on StackOverflow (with a much more readable LaTeX output). His solution is based on the ... [Read more...]

Sweave is something of a gold standard in reproducible research. It creates a dynamic document, written in a mix of LaTeX and R code where the results of the analysis (numbers, figures, tables) are automatically generated from the code and inserted into the resulting pdf document, making them easy to ... [Read more...]

In January of this year I first saw mention of the Julia language in the release notes for LLVM. I mentioned this to Dirk Eddelbuettel and later we got in contact with Viral Shahregarding a Debian package for Julia. There are many aspects of Julia tha... [Read more...]

Has Ireland gotten any warmer? Ask any punter on the street and they will happily inform you of wild swings, trends and dips. “Back when I was a child”, “when I was younger”, or “years ago” are the usual refrains. What’s the evidence? To answer this, I will use ... [Read more...]

The last Friday, March 30th, the last stable version of R, the version 2.15.0 was released.So, to install it in Debian Squeeze, or in another Distro powered by Debian (I actually use CrunchBang Linux), just follow the same instructions described here f... [Read more...]

The competition amongst database vendors to create the fastest, most powerful "data layer" — the hardware and software to provide storage for Big Data with high-performance data processing — is clearly heating up. The Netezza appliance has been so successful that IBM has been racing to keep up with demand. SAP is ... [Read more...]

There were lots of interesting developments this month that didn't work their way into a full blog post. Here is an incomplete list of what I've been tweeting about over the last few weeks. But first I want to draw your attention to the latest manuscri... [Read more...]

We love them, we hate them: RANKINGS!Rankings are an inevitable tool to keep the human rat race going. In this regard I'll pick up my last two posts (HERE & HERE) and have some fun with it by using it to analyse R-Bloggers' web presence. I will use... [Read more...]

Following the two previous posts (here and there), on the number of people that don't get wet while playing with water pistols, consider now an indoor version, in a non-convex room (i.e. player behind wall are now, somehow, protected). In the previ... [Read more...]

For some time I've contemplated creating a function for creating the dynamite plots beloved by many of the applied sciences. There's a lot of criticism regarding their utility, and there are several ways that present data in a more intelligible way. A search on the subject brings up pages with ... [Read more...]

[Read more...]

All around the world at noon GMT on April 28, data scientists around the world will compete in the world's first one-day International Data Science Hackathon, organized by Data Science London. Participants will receive a data set at the beginning of the event, and work in teams of 3-5 over the ... [Read more...]

A few weeks back I gave a talk at the local Berkeley R meetup group. The idea was to help people not make the same mistakes I made when I first started out learning R. It was the first time I made an entire presentation with Deck.js and I ... [Read more...]