NYT uses R to map the 1%

January 17, 2012
By
NYT uses R to map the 1%

Last Saturday, the New York Times published a feature article on the wealthiest 1% of Americans. The on-line version of the article included interactive features like this interactive map showing where your household ranks in the country and in local regions. The print edition, however, included some different (and necessarily static) representations of US wealth data, such as this...

Read more »

Time Based Arbitrage Opportunities in Tick Data

January 17, 2012
By
Time Based Arbitrage Opportunities in Tick Data

I recently posted an introduction to the Kaggle Algorithmic Trading Challenge, which I competed in.I said that I would post about my experiences, and this is hopefully the first of a series. We were given tick data from the London Stock Exchange(specifically, the FTSE 100) over random time intervals during parts of 37 days. Each data row...

Read more »

R-Code Yahoo Finance Data Loading

January 17, 2012
By

Quantitative Finance, Technical Trading & Analysis. Fotis Papailias, Dimitrios Thomakos Fotis Quantitative Finance & Technical Trading R-Code Yahoo Finance Data LoadingHere is an R script that downloads Yahoo Finance Data without the need of additional packages/libraries. In the .zip file is the code with an example on how to use it. Download the code here: You can also...

Read more »

Parallel R Loops in Windows and Linux

January 17, 2012
By

Parallel computation may seem difficult to implement and a pain to use, but it is actually quite simple to use. The foreach package provides the basic loop structure, which can utilize various parallel backends to execute the loop in parallel. First, let's go over the basic structure of a foreach loop. To get the foreach package, run the following...

Read more »

How to map geographically-detailed survey responses?

January 17, 2012
By

David Sparks writes: I am experimenting with the mapping/visualization of survey response data, with a particular focus on using transparency to convey uncertainty. See some examples here. Do you think the examples are successful at communicating both local values of the variable of interest, as well as the lack of information in certain places? Also, The post How...

Read more »

Mid-January flotsam: teaching edition

January 17, 2012
By
Mid-January flotsam: teaching edition

I was thinking about new material that I will use for teaching this coming semester (starting the third week of February) and suddenly compiled the following list of links: William Briggs writes It is time to stop teaching Frequentism to … Continue reading →

Read more »

R Ec2

January 17, 2012
By

Running R in the cloud So you want to run R in the cloud so you can set your Gibbs sampling off, forget about it, and not be paranoid about power cuts and reboots. Andrew Gelman hosted a good debate on the pros and cons of R in the cloud on his blog. ...

Read more »

Backtesting a Trading Strategy

January 16, 2012
By
Backtesting a Trading Strategy

I've ordered Time Series Analysis and Its Applications: With R Examples (Springer Texts in Statistics) to help me up the time series in R learning curve. So far what I have seen it looks good. The author has a good page with the issues in R and time ...

Read more »

R jumps from 25 to 19 in annual TIOBE rankings of programming language popularity

January 16, 2012
By
R jumps from 25 to 19 in annual TIOBE rankings of programming language popularity

The TIOBE index ranks popularity of programming languages according to their prevalence on the web. Back in February last year, the R language had risen to #25 in the charts, overtaking both SAS and Matlab. Earlier this month, TIOBE published its annual rankings of programming language popularity for 2011 and R has risen once again: it now ranks #19...

Read more »

A slice of S&P 500 skewness history

January 16, 2012
By
A slice of S&P 500 skewness history

How symmetric are the returns of the S&P 500? How does the skewness change over time? Previously We looked at the predictability of kurtosis and skewness in S&P constituents.  We didn’t see any predictability of skewness among the constituents.  Here we look at skewness from a different angle. The data Daily log returns of the … Continue reading...

Read more »

CRdata.org to shut down?

January 16, 2012
By

If you’re one of the R-bloggers or useRs, most probably you had heard about Crdata.org. In the early day, they are two very R related cloud computing services, one is CloudNumbers, another is CrData.org. Recently, we (may) received an email by Hamid ...

Read more »

Project Euler in R: Problem 23

January 15, 2012
By

I was just looking through the programming language statistics on Project Euler. It shows that only 7% of the problems have been solved in R, whereas 8% have been solved on any kind of spreadsheet.  This is outrageous!Let's look at the solution of...

Read more »

Scraping table from any web page with R or CloudStat

January 15, 2012
By
Scraping table from any web page with R or CloudStat

Scraping table from any web page with R or CloudStat: You need to use the data from internet, but don’t type, you can just extract or scrape them if you know the web URL. Thanks to XML package from R. It provides amazing readHTMLtable() function. For...

Read more »

Crawling facebook with R

January 15, 2012
By
Crawling facebook with R

So, let's crawl some data out of facebook using R. Don't get too excited though, this is just a weekend whatif project. Anyway, so for example, I want to download some photos where I'm tagged. First, we need an access token from facebook. I don't know how to get this programmatically, so let's get one manually, log on to facebook...

Read more »

Statistical Methods for the Chain Ladder Technique Revisited

January 15, 2012
By
Statistical Methods for the Chain Ladder Technique Revisited

Statistical Methods for the Chain Ladder Technique Revisited: Source: Statistical Methods for the Chain Ladder Technique Demo Background Forecasting outstanding claims and setting up suitable reserves to meet these claims is an important part of the b...

Read more »

R-squared for multilevel models

January 15, 2012
By

Fred Schiff writes: I’m writing to you to ask about the “R-squared” approximation procedure you suggest in your 2004 book with Dr. Hill. I’m a media sociologist at the University of Houston. I’ve been using HLM3 for about two years. Briefly about my data. It’s a content analysis of The post R-squared...

Read more »

Merging two data.frame objects while preserving the rows’ order

January 15, 2012
By
Merging two data.frame objects while preserving the rows’ order

Merging two data.frame objects in R is very easily done by using the merge function. While being very powerful, the merge function does not (as of yet) offer to return a merged data.frame that preserved the original order of, one of the two merged, data.frame objects. In this post I describe this problem, and offer Read more...

Read more »

R and MODFLOW

January 15, 2012
By
R and MODFLOW

Here are some functions for reading and writing MODFLOW files from R. I hope to update this in the future!################################################################################### read.modflow.pval ############################################...

Read more »

Big media waking up to big data

January 14, 2012
By
Big media waking up to big data

A recent Globe and Mail column points out that by 2018 in the United States alone there will be a shortfall of 190,000 specialists with deep analytical talent. It is good to see that the mainstream media is waking up to the need for applied training in data analytics. ...

Read more »

Moving window filters and the pracma package

Moving window filters and the pracma package

In my last post, I discussed the Hampel filter, a useful moving window nonlinear data cleaning filter that is available in the R package pracma.  In this post, I briefly discuss this moving window filter in a little more detail, focusing on two important practical points: the choice of the filter’s local outlier detection threshold, and the question of...

Read more »

Welcome Back, Me

January 14, 2012
By

It's been a few weeks since I last posted.  Sorry about that.  Unfortunately, sometimes you come home from work just not wanting to look at a computer.I'm working on a series of posts requested by a few friends.  They would like to see m...

Read more »

Prediction model with HANA and R

January 14, 2012
By
Prediction model with HANA and R

These days, I have been reading and playing a lot with R, and I really come to love it...of course, I don't have a clue on those weird statistics formulas, but it doesn't mean I can't use R and try do some awesome stuff with it.So, yesterday I was thin...

Read more »

Managing change

January 14, 2012
By
Managing change

Why the old and new the need to share time togetherIt takes time to appreciate the new. Even if the new is much better than the old. But, it is easy to forget when you yourself created the exciting new.At the end of August 2011 Google announced a new B...

Read more »

Staying up with R

January 13, 2012
By
Staying up with R

No, I don’t mean late night coding. R is constantly changing – both as a language and a platform. Updates containing new functionality are frequent. New and revised packages appear several times a week.  Staying current with these myriad changes can … Continue reading →

Read more »

Are We Japanese?

January 13, 2012
By
Are We Japanese?

Most of the discussion trying to determine if the U.S. is Japan 20 years later focuses on the economy and the stock market.  However, one of the biggest and most persistent correlations between Japan and the U.S. are the Japanese Yen and the U.S. ...

Read more »

Revolution Newsletter: January 2012

January 13, 2012
By

The most recent edition of the Revolution Newsletter is out. The news section is below, and you can read the full January edition (with highlights from this blog and community events) online. You can subscribe to the Revolution Newsletter to get it monthly via email. Reflections on 2011. Last year was a big one for Revolution Analytics, with new...

Read more »

Time Series Matching

January 13, 2012
By
Time Series Matching

THIS IS NOT INVESTMENT ADVICE. The information is provided for informational purposes only. If it looks like a duck, swims like a duck, and quacks like a duck, then it probably is a duck. Do you want to know what S&P 500 will do in the next week, month, quarter? One way to make an

Read more »

Retrieving RSS Feeds Using Google Reader

January 13, 2012
By
Retrieving RSS Feeds Using Google Reader

I have been working on a new package makeR to help manage Sweave projects where you wish to create multiple versions of documents that are based on a single source. For example, I create lots of monthly and quarterly reports using Sweave and the only differences between versions are a few variables. I have used GNU

Read more »

R: A Quick Scrape of Top Grossing Films from boxofficemojo.com

January 13, 2012
By
R: A Quick Scrape of Top Grossing Films from boxofficemojo.com

  Introduction I was looking at a list of the top grossing films of all time (available from boxofficemojo.com) and was wondering what kind of graphs I would come up with if I had that kind of data. I still don’t know what kind of graphs I’d construct other than a simple barplot but figured

Read more »