While reading up on quantile regression I found a really nice hack described in Bayesian Quantile Regression Methods (Lancaster & Jae Jun, 2010). It is called Jeffreys’ substitution posterior for the median, first described by Harold Jeffreys in his Theory of Probability, and is a non-parametric method for approximating the posterior of ... [Read more...]

Following my last post on decision making trees and machine learning, where I presented some tips gathered from the "Pragmatic Programming Techniques" blog, I have again been impressed by its clear presentation of strategies regarding the evaluation of model performance. I have seen some of these topics presented elsewhere - ... [Read more...]

The very useful Valgrind tool had flagged an actual error in the package which the CRAN maintainers asked us to address. This was followed by a minor brown-bad oversight of a missing delete, also tagged by Valgrind. Both are pretty ancient bugs whic... [Read more...]

In my paper on the impact of the shale oil and gas boom in the US, I run various instrumental variables specifications. For these, it is nice to stack the regression results one on the other – in particular, to have one row for the IV results, one row for the ... [Read more...]

Today, I want to follow up with the Calendar Strategy: Month End post. Let’s examine the perfromance Option Expiry days as presented in the The Mooost Wonderful Tiiiiiiime of the Yearrrrrrrrr! post. First, I created two convenience functions for creating a calendar signal and back-testing calendar strategy: calendar.signal ... [Read more...]

Test coverage of the 10 most downloaded R packages 2014-04-30 Source Introduction How do you know that your code is well tested ? The test coverage is the proportion of source code lines that are executed (covered) when running the tests. It is useful to find the parts of your code ... [Read more...]

Ok, I confess. Until yesterday I was one of those still trying to read large data tables using read.table. Then, I came across this thread in stackoverflow and I saw the light. Since I noticed that a lot of people still struggle with read.table I decided to write ... [Read more...]

I'll say it again: the 2048 game is horribly addictive. Luckily for my productivity, though, I've been playing a lot less of it since I finally managed to complete the game's main goal: combine and double-up like-numbered tiles until you create the ultimate 2048 tile: My strategy? Play the game like Tetris. ... [Read more...]

This page shows Multidimensional Scaling (MDS) with R. It demonstrates with an example of automatic layout of Australian cities based on distances between them. The layout obtained with MDS is very close to their locations on a map. At first, … Continue reading → [Read more...]

This is not nearly as interesting as it might first sound, but every function in R contains R code; this is true of core R code as well as extension packages. Sometimes the R code is just a very shallow wrapper around some compiled code, such as in sum() and ... [Read more...]

Great news! The RSS is setting a data analysis challenge this year, sponsored by the Young Statisticians Section and Research Section of the Royal Statistical Society: Details are available on the wordpress website of the Challenge. Registration is open and the Challenge goes live on Tuesday 6 May 2014 for an exciting 6 ... [Read more...]

The goal of Zumel/Mount: Practical Data Science with R is to teach, through guided practice, the skills of a data scientist. We define a data scientist as the person who organizes client input, data, infrastructure, statistics, mathematics and machine learning to deploy useful predictive models into production. Our plan ... [Read more...]

by Joseph Rickert Tuesday's post on a new Kaggle contest mentioned that Revolution Analytics offers a free trial for using Revolution R Enterprise in the Amazon cloud. One reason this might be of interest to contestants is the rxImport() function which reads delimited text data, fixed format text data, and ... [Read more...]

I recently wrote about my workflow for Analyzing Microbial Growth with R. Perhaps the most important part of that process is the plate map, which describes the different experimental variables and where they occur. In the example case, the plate map described which strain was growing and in which environment … [Read more...]

So day 1 of the #JHUSMARTHack 2014 is over and day 2 is underway. It's been awesome. One student estimated he did “one week's worth of work”. Some interesting (I think) things I thought about: Flooding rain is good weather for coding. (As long as no sinkholes arise) The Admiral Fell Inn is ... [Read more...]

Programming New Things I currently have been programming in R for more than half a decade and can now fondly look back on the days when I went on a spring break with Venables' little blue book (now out of print).  Back then I was entertained and wowed by R’... [Read more...]

I’ve been doing a lot of analyses recently that need rasters representing features in the landscape. In most cases, these data have been supplied as shapefiles, so I needed to quickly extract parts of a shapefile dataset and convert them … Continue reading → [Read more...]

[Read more...]