I had someone ask me the other day how to take a scatterplot and draw something other than a straight line through the graph using Excel.  Yes, it can be done in Excel and it’s really quite simple, but there are some limitations when using the stock Excel dialog ... [Read more...]

Previously, I came up with a solution to R's less than ideal handling of named capture in regular expressions with my re.capture() function. A little more than a year later, the problem is rearing its ugly - albeit subtly different - head again. I now... [Read more...]

I wanted to get back on some interesting experience, following a discussion I had with Carlos after my class, this morning. Let me simplify the problem, and change also the dataset. Consider the following dataset __ db = read.table("http://freakonometrics.free.fr/db2.txt",header=TRUE,sep=";") Let me change ... [Read more...]

This is mostly for my students and myself for future reference.Classification is a supervised task , where we need preclassified data and then on new data , I can predict.Generally we holdout a % from the data available for testing and we call them tra... [Read more...]

I have immersed myself in polar coordinates.  Here is another little reconstruction that I did. [Read more...]

We're thrilled to have John Wallace and Tess Nesbitt from DataSong join our Fall webinar series tomorrow, with a great presentation on time to event models. If you're trying to predict when an event will occur (for example, a consumer buying a product) or trying to infer why events occur (... [Read more...]

I am currently a research fellow and 4th year PhD candidate within the INLA group.  If you deal with Bayesian models and have never heard about INLA, I sincerely think you should spend a small portion of your time to at least know what it is. If you have heard ... [Read more...]

Let us continue our discussion on smoothing techniques in regression. Assume that . where is some unkown function, but assumed to be sufficently smooth. For instance, assume that  is continuous, that exists, and is continuous, that  exists and is also continuous, etc. If  is smooth enough, Taylor’s expansion can be ... [Read more...]

This nice little tool Raw from DensityDesign transforms text from your clipboard to d3.  For those yearning to access Raw from R, here is an easy way to do it. Use this function read.excel from StatisticallySignificant’s post Copying Data from Excel to R and Back.  Once you run ... [Read more...]

In a standard linear model, we assume that . Alternatives can be considered, when the linear assumption is too strong. Polynomial regression A natural extension might be to assume some polynomial function, Again, in the standard linear model approach (with a conditional normal distribution using the GLM terminology), parameters can be ... [Read more...]

Tableau, the popular interactive data visualization tool, is coming out with a new 8.1 update, and it will include integration with the R language. Access to R is a feature that has been requested by Tableau users for some time, and was met with rapturous applause when it was announced at ... [Read more...]

The previous post outlined the general strategy of writing a MH within Gibbs sampler by breaking the code into two levels: a high level shell and a series of lower-level samplers which do the actual work. This post discusses the … Continue reading → [Read more...]

The new micropolar library from Chris Viau, author of   gives us reusable charts with polar coordinates for d3.js.  In testament to rCharts design, Ramnath Vaidyanathan integrated micropolar with rCharts in less than 30 minutes.  I don’t use polar coordinates much, so I wrote this quick little comparison of Cartesian ... [Read more...]

In the chapter, we argue that a useful way to develop a Metropolis-Hastings (MH) within Gibbs sampler is to split the code into two levels. The top level is the "shell" of the MH within Gibbs algorithm, which sets up … Continue reading → [Read more...]

The key to programming is being lazy; it has actually been called a virtue by some. When I discovered the update() function it blew me away. Within short I had created a monster based upon this tiny function, allowing quick and easy output of regression tables that contain crude and ... [Read more...]

I decided to teach a basic course on bayesian data analysis with a focus on psycholinguistics. Here is the course website (below). How could this possibly be a bad idea!http://www.ling.uni-potsdam.de/~vasishth/advanceddataanalysis.html [Read more...]

Note: This is cross-posted from the rOpenSci blog, which will update with this post when our technical snafu is fixed. With the US government shut down, many of the federal government provided data APIs are down. We write R packages to interact with m... [Read more...]

Read away, A interesting post about skills to become Data Scientist.The post is about  Where to start? When do you start seeing light at the end of the tunnel? What is the learning roadmap? What tools and techniques do I need to know? &n... [Read more...]

The population of Native Americans in the United States is estimated to be anywhere upto 20 million (http://en.wikipedia.org/wiki/Native_Americans_in_the_United_States). I wanted to find out what the population of Native Americans (originally referred to as American Indians by immigrants) is and compare it ... [Read more...]

I’ve made quite a few blog posts about neural networks and some of the diagnostic tools that can be used to ‘demystify’ the information contained in these models. Frankly, I’m kind of sick of writing about neural networks but I wanted to share one last tool I’ve ... [Read more...]