Due largely to the work of Clint Cummins, the sas7bdat file format has become a bit less shrouded. In particular, we now know the following: how to detect files with compressed data (and fail graciously) more details about the platform that generated the file (e.g., endianess, OS details) ... [Read more...]

Having just lived through two back to back hurricanes (Irene in 2011 and Sandy in 2012) that passed through the New York metro area I was curious how the paths of the hurricanes differed.  I worked up a quick graph in R using data from Unisys.  The data also includes wind speed ... [Read more...]

Lately I have been working on a trading system based on Support Vector Machine (SVM) regression (and yes, if you wonder, there are a few posts planned to share the results). In this post however I want to share an interesting problem I had to deal with. Few days ago, ... [Read more...]

Bootstrapping is a very popular statistical technique. However, its Bayesian analogue proposed by Rubin (1981) is not very common. I was looking for an example of its implementation in GNU R and could not find one so I decided to write a snippet presen... [Read more...]

This is the question that Hadley Wickham recently set out to discovering by asking frequent R and plyr users how they use it in an online survey. Once a decent number of [Read more...]

[Read more...]

Ryan Peek showed us how to use an .Rprofile file to customize your R setup. Here are his instructions and script: For Windows To change profile for R, go here: C:\Program Files\R\R-2.15.1\etc (or whatever version you are using) Edit the “Rprofile.site” file Restart R For ... [Read more...]

I had a great time presenting my new webinar yesterday, thanks to everyone who attended "The Rise of Data Science in the Age of Big Data Analytics" and especially those who submitted questions. Sorry I didn't have time to get to them all, but feel free to ask here in ... [Read more...]

New Madrid, Territory of Missouri, March 22, 1816 Dear Sir, In compliance with your request, I will now give you a history, as full in detail as the limits of the letter will permit, of the late awful visitation of Providence in this place and vicinity.  On the 16th of December, 1811, about ... [Read more...]

From time-to-time creating a basic map of the United States or other parts of the world to complement some statistical analysis is useful to emphasize a point. The maps package in R provide a good way to produce these these maps.  These maps axes are based on latitude and longitude ... [Read more...]

So, continuing with the short tutorials on how to do relatively simple (but sometimes very frustrating things) in ggplot, today’s post looks at how to use gradient colors and Brewer colors to color either continuous or discrete dependent variab... [Read more...]

Security and R One of the more challenging aspects of OpenCPU is security in R (or lack thereof). This is actually one of the reasons OpenCPU runs on Linux only at this point; other operating systems simply lack superpowers to implement open computing. (Maybe one exception is BSD, for which ... [Read more...]

When it comes to R blogging I'm a complete newbie. So I'm still struggling with the technical details.Part of the process is prettifying the code snippets. One of the standard ways of doing this involves copy-and-paste-ing the R code into the Pretty R ... [Read more...]

I've been alluding to different R data types, or classes, in various posts, so I want to go over them in more detail. This is part 1 of a 3 part series on data types. In this post, I'll describe and give a general overview of useful data types.  I... [Read more...]

The Washington Post has an interactive graphic showing the rate at which the US presidential candidates Barack Obama and Mitt Romney have visited the various states for campaign rallies and fundraisers. Here's how it looks today: You can clearly see the focus on key swing states like Florida and Ohio, ... [Read more...]

Here is the list of press reports and news about R Bits (A bog under The New York Times) R you ready for R? by Ashlee Vance Published: January 8, 2009, 1:52 PM The New York Times Data Analysts Captivated by R’s Power by Ashlee Vance Published: January 6, 2009  InfoWorld The BI battle ... [Read more...]

I am working on a project that requires the generation of Bernoulli outcomes. Typically, I would go about this using the built in sample() function like so: This works great and is fast, even for large n. Problem is, I want to generate each sample with its own unique probability. ... [Read more...]

I recently had a task to take a look at some assessment (audit) data. I was assuming, rather hoping for data with a normal distribution and thought it would be a quick case of Pearson correlation between two columns: "Duration" and "Score". Just conjecture at this point as I did ... [Read more...]

(By Hadley Wickham) Hi all, I’d like to let you know about four R training courses that RStudio will be offering in December: * Effective data visualization [Read more...]

So far we’ve covered Melting and Casting data using the reshape() package and today we’re going to look at different ways of coloring and selecting palettes for plots. For these plots, we’re going to use the built in diamonds data... [Read more...]