In the 2012 edition of the SAP Sybase Capital Markets Guide, Revolution Analytics' Senior Advisor for Products and Strategy (and former CEO) Norman Nie writes about the "Five Benefits of Big Analytics". (You can also read his article at Enterprise Innovation.) Norman makes the argument that while sampling and aggregation are ... [Read more...]

[Read more...]

The first thing, what you have to do is to estimate how many users will be interested in local R group. I would say, that out of one million inhabitants you can expect 10-20 users. Based on this raw number, you can know, what challenges are waiting for you. If ... [Read more...]

Ok so I have now done two iterations on a better way to visualize term frequencies using R, ggplot2 and plyr. The first was ok but ugly, the second was better but still ugly. How to read it: Frequency is segmented in to 20% quantiles The frequency is on the y ... [Read more...]

The Penn World Tables are one of the greatest source of worldwide macroeconomic data, but dealing with its web interface is somewhat cumbersome. Fortunately, the data is also available as a R package on CRAN. Having some tools at hand … Continue reading → [Read more...]

[Read more...]

The ff package is great when you are working with large data in R. Data in corporate environments are usually not that large that a Hadoop system is needed to handle it but the data are mostly large enough to make R choke on it's RAM.  T... [Read more...]

Stewart Biology Building, McGill University (Rm N4/17) Monday, April 24, 2012  14h-16h Dr. Arthur Charpentier (UQàM) In this workshop we will examine difference concepts related to quantiles, and practical issues based on R codes. This workshop will present quantile regression, and the idea of iterative least square estimation. It will ... [Read more...]

I've been working on different projects lately and my time for this blog, unfortunately, has been close to zero. But that's going to change. Don't expect new post every day, but there should be a new post at least in every two weeks. Anyway, let's get back to the point ... [Read more...]

The webinar team at Revolution Analytics has put together a great program over the next couple of months. With a mix of guest speakers and Revolution Analytics staff, this series will cover topics as diverse as Big Data with R and Hadoop, integrating R with MS Office, spatial statistics with ... [Read more...]

Cognitive Behavioral Therapy is the psychological treatment of choice for many, if not all, mental disorders. Nonetheless a majority of US clinical psychologist do not primarily identify themselves as either cognitive or behavioral therapists. Looking at data from PubMed publication counts a clear picture emerges; psychodynamic researchers might just be ... [Read more...]

The mixing matrix of a graph gives the density of edges between vertices with different characteristics. The mixing matrix for a given igraph object can be calculated using the following function: The assortativity coefficient, based on Newman’s paper, can be … Continue reading → [Read more...]

[Read more...]

Here is an alternative to word clouds that makes it easier to get insights, but also has some of the aesthetic appeal of the traditional word cloud. My first attempt at this looked pretty bad and this is not too much better, but hopefully someone else will help improve it. ... [Read more...]

[Read more...]

Sometimes the basic functionality in lattice graphics isn't enough. Custom "panel functions" are one approach to fully customizing the lattice graphics system. Two examples are given below illustrating how to define an (inline) custom panel function fo... [Read more...]

Timely Portfolio has a nice post about linear models sytems for stock. The idea follows from the steps below: Get the weekly closing values of the S&P 500. Choose a time window (i.e. 25 weeks) and for each window, linearly regress the subset of closing values Choose an investment strategy ... [Read more...]

Thanks to the lower-than-usual snowfall over most of the US this past winter, there's low risk of major flooding as the snow melts this Spring (for the first time in four years!). Nonetheless, being able to forecast river flood events is of critical importance to local emergency managers, water & electric ... [Read more...]

To celebrate the beginning of the professional baseball season here in the US and Canada, we revisit a famous example of using baseball data to demonstrate statistical properties. In 1977, Bradley Efron and Carl Morris published a paper about the Jame... [Read more...]

Here is a quick quiz. If you visit the Wikipedia page List of countries by GDP, you will find three lists ranking the countries of the world in terms of their Gross Domestic Product (GDP), each list corresponding to a different source of the data. If you pick the list ... [Read more...]