Bill Bolstad's response to Xi'an's review of his book Understanding Computational Bayesian Statistics included the following comment, which I found interesting: Frequentist p-values are constructed in the parameter dimension using a probability distribution defined only in the observation dimension. Bayesian credible intervals are constructed in the parameter dimension using a ... [Read more...]

Exploratory factor analysis (EFA) is a common technique in the social sciences for explaining the variance between several measured variables as a smaller set of latent variables. EFA is often used to consolidate survey data by revealing the groupings ... [Read more...]

[Read more...]

For those who naturally compute portfolio returns correctly here are some lessons in how to do it wrong. The data Random portfolios were generated from constituents of the S&P 500 with constraints: long-only exactly 20 assets in the portfolio no more than 10% weight for any asset (just for fun) the sum ... [Read more...]

Exercise 4 required implementing Logistic Regression using Newton's Method. The dataset in use is 80 students and their grades of 2 exams, 40 students were admitted to college and the other 40 students were not. We need to implement a binary classification model to estimates college admission based on the student's scores on these two ... [Read more...]

As a follow-up to my isarithmic maps of county electoral data, I have attempted to experiment with extending the technique in two ways. First, where the electoral maps are based on data aggregated to the county level, I have sought to generalize the method to accept individual responses for which ... [Read more...]

Part 2: Autocomplete with YUI3 and GAE RPC HandlersThis is second and final part of 2-part series on implementing AutoComplete with GAE. Recall that in part 1 we built a foundation for keyword autocomplete lookup service for our document search application. Both the service itself and its HTML/JavaScript client will materialize ... [Read more...]

Last week a question came up on Stack Overflow about determining whether a variable is distributed normally. Some of the answers reminded me of a common and pervasive misconception about how to apply tests against normality. I felt the topic was general enough to reproduce my comments here (with minor ... [Read more...]

How was the Netflix Prize won? I went through a lot of the Netflix Prize papers a couple years ago, so I’ll try to give an overview of the techniques that went into the winning solution here. Normalization of Global Effects Suppose Alice rates Inception 4 stars. We can think ... [Read more...]

Bill Bolstad wrote a reply to my review of his book Understanding computational Bayesian statistics last week and here it is, unedited except for the first paragraph where he thanks me for the opportunity to respond, “so readers will see that the book has some good features beyond having a “... [Read more...]

In my last few posts, I have been discussing some of the consequences of the slow decay rate of the tail of the Pareto type I distribution, along with some other, closely related notions, all in the context of continuously distributed data.  Today’s post considers the Zipf distribution for ... [Read more...]

[Read more...]

[Read more...]

My previous post talked about how we can employ PCA on the data for multiple stock returns to reduce the number of variables in explaining the variance of the underlying data. But the idea was greeted with skepticism by many. A caveat to the applicatio... [Read more...]

In our last tutorial on SVM training with GPU, we mentioned a necessary step to pre-scale the data with rpusvm-scale, and to reverse scaling the prediction outcome. This cumbersome procedure is now simplified with the latest RPUSVM. read more [Read more...]

Part 1: Embedded Relation Index EntityI continue series of posts about keyword-based searches on Google App Engine (GAE) using Relation Index Entity (see RIE with Java and REI with Python posts in this order). Having implemented efficient search on GAE let's switch the focus to usability. When user searches for a ... [Read more...]

This binary package supports R 2.13.x (32-bit/64-bit) and MySQL 5.5.16 (32-bit/64-bit). RMySQL 0.8-0 for MySQL 5.5.16 [Read more...]

Rose Hoffmann, AP Statistics teacher at Catholic Memorial High School in Waukesha, WI sent the following note to the Revolution Analytics team: In August 2010, my husband who is a statistician attended the American Statistical convention. Your company gave out the flying monkey with a black cape ... He gave me the ... [Read more...]

There are several blog posts, websites (and even books) explaining the transition from using another statistical system (e.g. SAS, SPSS, Stata, etc) to relying on R. Most of that material treats the topic from the point of view of i- … Continue reading → [Read more...]

(Hadley Wickham, author of ggplot2 and several other R packages, guest blogs today about forthcoming big-data improvements to his R graphics package -- ed.) Hi! I'm Hadley Wickham and I'm guest posting on the Revolutions blog to give you a taste of some of the visualisation work that my research ... [Read more...]