Much of my time lately has gone into analyzing and trading products in the volatility complex.  As a result, I regularly watch the VIX term structure for continuations or deviations from trend.  To make analysis simpler, I’ve written some… Read more › [Read more...]

Dear R experts, I sent this question to the r-help list but didn’t get much response, probably because it is more of a stats question. But as this blog is syndicated on r-bloggers I thought I would try it again here on this blog. If I am barking up ... [Read more...]

This post is a little more esoteric than most, but I found myself needing to solve this problem, so I’m just passing the solution on to you. The plot above shows the distribution of DW-NOMINATE scores for the 18th Congress, with party indicated ... [Read more...]

[Read more...]

Here’s an addition to my last post on using Wikipedia data to analyse attention for the US presidential elections 2012. Here’s another look at the interest not for the candidates’ Wikipedia pages but the general pages for the elections 2008 and 2012. Compared to the candidates’ pages, the attention for the ... [Read more...]

  In this post I like to illustrate the R package “ape” for phylogenetic trees for the purpose of assembling trees. The function read.tree creates a tree from a text description. For example the following code creates and displays two … Continue reading → [Read more...]

For now this is the last post about these football data. It started in August, by now it is November. But just to finish up; the model as it should have been last week.ModelAs most of what I did is described last week, only the model as it went ... [Read more...]

Brian Caffo headlines the WaPo article about massive online open courses. He is the driving force behind our department’s involvement in offering these massive courses. I think this sums it up: `“I can’t use another word than unbelievable,” Caffo said. … Continue reading → [Read more...]

Plotting large amounts of hourly atmospheric data body, td { font-family: sans-serif; background-color: white; font-size: 12px; margin: 8px; } tt, code, pre { font-family: 'DejaVu Sans Mono', 'Droid Sans Mono', 'Lucida Console',... [Read more...]

One of the most interesting challenges of data science are predictions for important events such as national elections. With all those data streams of billions of posts, comments, likes, clicks etc. there should be a way to identify the most important correlations to make predictions about real-world behavior such as: ... [Read more...]

My previous post is about a method to simulate a Brownian motion. A friend of mine emailed me yesterday to tell me that this is useless if we do not know how to simulate a normally distributed variable. My first remark is: use the rnorm() function if t... [Read more...]

A few days ago a post doctoral researcher asked me if I could help him reorder the factor levels on a bar chart. The problem is that R automatically alphabetizes factor levels. I thought this would be fairly straight-forward but... [Read more...]

It’s been over one month since my last post on Euler problem 20, when  I was planning to post at least one on either Euler project or visualization. So I am four posts behind; I’ll try to catch up. Tonight, I’ll solve the 21st Euler … Continue reading → [Read more...]

A real developer never stops learning that's a quote I always love to repeat...because it applies to my life...you can know a lot of things but there's always something new to learn, or to re-learn. That's why a couple of days ago I start reading wxPyt... [Read more...]

Due largely to the work of Clint Cummins, the sas7bdat file format has become a bit less shrouded. In particular, we now know the following: how to detect files with compressed data (and fail graciously) more details about the platform that generated the file (e.g., endianess, OS details) ... [Read more...]

Having just lived through two back to back hurricanes (Irene in 2011 and Sandy in 2012) that passed through the New York metro area I was curious how the paths of the hurricanes differed.  I worked up a quick graph in R using data from Unisys.  The data also includes wind speed ... [Read more...]

Lately I have been working on a trading system based on Support Vector Machine (SVM) regression (and yes, if you wonder, there are a few posts planned to share the results). In this post however I want to share an interesting problem I had to deal with. Few days ago, ... [Read more...]

Bootstrapping is a very popular statistical technique. However, its Bayesian analogue proposed by Rubin (1981) is not very common. I was looking for an example of its implementation in GNU R and could not find one so I decided to write a snippet presen... [Read more...]

This is the question that Hadley Wickham recently set out to discovering by asking frequent R and plyr users how they use it in an online survey. Once a decent number of [Read more...]

[Read more...]