The latest slides from web scraping through R: Web scraping for the humanities and social sciencesSlides from the first session hereSlides from the second session hereThis week we look in greater detail at scaling up digital data-collection: coercing s... [Read more...]

Introduction Error propagation can be a fair bit of work with a calculator, but it’s easy with R. Just use R in repeated calculation with perturbed quantities, and inspect the range of results. Two methods are shown below for inspecting the range: sd() and quantile(), the latter using the ... [Read more...]

Python has an extremely intuitive and straightforward way of dealing with APIs, and makes it simple for people like you or me to access and retrieve information from databases. Before I quickly describe how to use APIs in Python, maybe we should begin with: What is an API? API (Application ... [Read more...]

This is another situation where Fourier terms are useful for handling the seasonality. Not only is the seasonal period rather long, it is non-integer (averaging 365.25/7 = 52.18). So ARIMA and ETS models do not tend to give good results, even with a period of 52 as an approximation. Regression with ARIMA errors The ... [Read more...]

The release date for Zumel, Mount “Practical Data Science with R” is getting close. I thought I would share a few statistics about what goes into this kind of book. “Practical Data Science with R” started formal work in October of 2012. We had always felt the Win-Vector blog represented practice ... [Read more...]

Two items of post from utility companies that recently dropped through our door included little graphics. There was a degree of innovation in them both. The first, from British Gas, is technically OK but probably bad on perceptual grounds: I got … Continue reading → [Read more...]

by Rodney Sparapani, PhD Rodney is an Assistant Professor in the Institute for Health and Society from the Division of Biostatistics at the Medical College of Wisconsin in Milwaukee and president of the Milwaukee Chapter of the ASA which is hosting an R workshop on Data Mining in Milwaukee on ... [Read more...]

The Star Puzzle is a puzzle presented on The Math Forum.  I became aware of this problem by noticing the article and solution posted on Quantitative Decisions article section. It asks the question, "How many triangles, quadrilaterals, and irregula... [Read more...]

This was the first day of our workshop Advances in Scalable Bayesian Computation and it sounded like the “main” theme was probabilistic programming, in tune with my book review posted this morning. Indeed, both Vikash Mansinghka and Frank Wood gave talks about this concept, Vikash detailing the specifics of a ... [Read more...]

First of all, I’d like to thank my readers for the lots of feedback on my last post on beautiful outputs in R. I tried to consider all suggestions, updated the existing table-output-functions and added some new ones, which will be described in this post. The updated package is ... [Read more...]

Last week's Cologne R user group meeting was all about R and databases. We had three talks from a generic overview on how to connect R to databases, to a specific example with kdb+ and perhaps the future with ArangoDB, a NoSQL database.Connecting R with databasesDiego de Castillo's talk ... [Read more...]

Following my post on fitting models to long time series, I thought I’d tackle the opposite problem, which is more common in business environments. I often get asked how few data points can be used to fit a time series model. As with almost all sample size questions, there ... [Read more...]

Since we've had quite a few announcements over the last month or so, I thought I'd take a moment to catch up on some of the media reports mentioning Revolution Analytics. Last week, Gartner revealed Revolution Analytics as a Visionary in the new Magic Quadrant for Advanced Analytics Platforms. Inside ... [Read more...]

There is now an excellent tutorial on using Quandl via R at DataCamp.com. Datacamp offers really well designed (and free) in-browser tutorials for learning more effective data analysis. Their focus thus far has been on R. The free interactive Quandl course introduces you to the main functionality in the ... [Read more...]

While preparing the practicals for our course at the University of Alberta, I've discovered something kind of interesting. I'm sure this is nothing new and actually people who normally use both OpenBUGS and JAGS have already figured this out.&nbsp... [Read more...]

Here's comes a script I wrote for creating contour lines in KML-format to be used with Google Earth https://github.com/gimoya/theBioBucket-Archives/blob/master/R/r_contours_for_google_earth.RIf you want to check or just use the datasets I created for t... [Read more...]

I don't usually like to use complex statistical methods, but every once in a while I encounter a method that is so useful that I can't avoid using it. Around the time I started doing eye-tracking research (as a post-doc with Jim Magnuson), people were ... [Read more...]

Apparently, this turned out to be my most popular post ever.  Of course there are lots of things to say about the heatmap (or quilt, tile, guilt plot etc), but what I wrote was literally just a quick celebratory post to commemorate that I’d finally grasped how to combine ... [Read more...]

The EARL (Effective Applications of the R Language) Conference takes place in London on the 16th and 17th September 2014. It will provide an opportunity for those using and developing the open source statistical programming language R to share and discuss their innovative practices of the R Language. The Conference will ... [Read more...]

In my previous blog post I have explained the steps needed to solve a data analysis problem. Going further, I will be discussing in-detail each and every step of Data Analysis. In this post, we shall discuss about exploratory Analysis.What is Exploratory Analysis?“Understanding data visually”Exploratory Analysis means ... [Read more...]