Consider the problem of making statistical inferences, as opposed to predictions. The product of statistical inference is a probabilistic statement about a population quantity, for example a 100(1-)% confidence interval for a population median. In this context, the principal reason for diagnostics is to comfort ourselves about the quality of ... [Read more...]

The Titanic challenge on Kaggle is about inferring from a number of personal details whether a passenger survived the disaster or did not. I gave two algorithms a try, which are decision trees using R package party and SVMs using … Continue reading → The post Titanic challenge on Kaggle with decision ... [Read more...]

Revolution Analytics has just introduced a 10-module series of R courses in Singapore. If you'd like to learn how to do data analysis in R, already know data analysis in another language like SAS and want to transition to R, or just want to enhance your R skills in a ... [Read more...]

Back in 2011, Charlie Park did two very thorough posts on Edward Tufte’s table graphics or slopegraphs. http://charliepark.org/slopegraphs/ http://charliepark.org/a-slopegraph-update/ These type graphics can provide very effective visualizations of ... [Read more...]

A article of mine was published in Science today. It introduces estimates for bilateral global migration flows between all countries. The underlying methodology is based on the conditional maximisation routine in my Demographic Research paper. However, I tweaked the demographic … Continue reading → [Read more...]

Blog post: a very basic introduction to building data plots using ggplot2 in R. #Rstats [Read more...]

A number of talks about Rcpp are scheduled over the next couple of months: Our sixth internation conference R/Finance, will take place May 16 and 17, 2014, at UIC in Chicago. See the current draft agenda for details of the program---whic... [Read more...]

In this post I will go through 5 reasons: zero cost, crazy popularity, awesome power, dazzling flexibility, and mind-blowing support. I believe R is the best statistical programming language to learn. As a blogger who has contributed over 150 posts in Stata and over 100 in R I have extensive experience with both ... [Read more...]

[Read more...]

Part I showed the concept and general technique of a method of assigning n email addresses to x cells pseudo-randomly, without the need for maintaining a log of each assignment.The earlier post considered the basic case of each cell being assigned approximately the same quantity of email addresses. In ... [Read more...]

This workshop is aimed at scientist/practitioners that are interested in analyzing complex datasets by fitting linear mixed models, particularly users with experience in [Read more...]

[Read more...]

The %.% operator in dplyr allows one to put functions together without lots of nested parentheses. The flanking percent signs are R’s way of denoting infix operators; you might have used %in% which corresponds to the match function or %*% which is matrix multiplication. The %.% operator is also called chain, and ... [Read more...]

Beauty is the first test; there is no permanent place in the world for ugly mathematics (G. H. Hardy) Newton basin fractals are the result of iterating Newton’s method to find roots of a polynomial over the complex plane. It maybe sound a bit complicated but is actually quite ... [Read more...]

Principal Components Analysis (PCA) is used as a dimensionality reduction method. Here we simply explain PCA step-by-step using data about Sochi Olympic Curlers. It is hard to visualize a high dimensional space. When I took linear algebra, the book and teachers spoke about it as if were easy to visualize ... [Read more...]

Parallel coordinates plot is one of the tools for visualizing multivariate data. Every observation in a dataset is represented with a polyline that crosses a set of parallel axes corresponding to variables in the dataset. You can create such plots in R using a function parcoord in package MASS. For ... [Read more...]

by Joseph Rickert Worldwide R user group activity for the first Quarter of 2014 appears to be way up compared to previous years as the following plot shows. The plot was built by counting the meetings on Revolution Analytics R Community Calendar. R users continue to value the live, in person ... [Read more...]

I messed around with R for years without really learning how to use it properly. I think it’s because I could always throw my hands up when the going got tough and run back and cling the skirts of Excel or JMP or Systat. I finally learned how to ... [Read more...]

A few days ago I posted about Filtering Data with L2 Regularisation. Today I am going to explore the other filtering technique described in the paper by Tung-Lam Dao. This is similar to the filter discussed in my previous post, but uses a slightly different objective function: where the regularisation ... [Read more...]

I just submitted my package update (version 1.3) to CRAN. The download is already available (currently source, binaries follow). While the last two updates included new functions for table outputs (see here and here for details on these functions), the current update only provides small helper functions as new functions. The ... [Read more...]