In econometric modeling, I usually have a problem with correlated features. A few weeks ago, I was discussing feature selection when features are correlated. This week, I was wondering about reverse engineering when features might be correlated (not to say very correlated). The way I see reverse engineering is the ...

In order to illustrate hierarchical clustering techniques and k-means, I did borrow François Husson‘s dataset, with monthly average temperature in several French cities. __ temp=read.table( + "http://freakonometrics.free.fr/FR_temp.txt", + header=TRUE,dec=",") We have 15 cities, with monthly observations __ X=temp[,1:12] __ boxplot(X) Since the ...

High resolution and SVG versions of the new R logo are finally available. I converted the SVG to WKT (file here) which means we can use it like we would a shapefile in R. That includes plotting! Here’s a short example of how to read that WKT and plot ...

So rock me mama like a wagon wheel, rock me mama anyway you feel (Wagon Wheel, Old Crow Medicine Show) This is the third iteration of Hilbert curve. I placed points in its corners. Since the curve has beginning and ending, I labeled each vertex with the order it occupies:...

<img src=' [Read more...]

One of the trickier tasks in clustering is determining the appropriate number of clusters. Domain-specific knowledge is always best, when you have it, but there are a number of heuristics for getting at the likely number of clusters in your data. We cover a few of them in Chapter 8 (available ...

This article examines FEC data in depth and finds what most people already know. Hillary Clinton's presidential bid is financed largely through a relatively small quantity of big donors while Bernie Sanders' presidential bid is funded by numerous small donors.In order to do our analysis, we look at four ...

When you visit a site like the LA Times’ NH Primary Live Results site and wish you had the data that they used to make the tables & visualizations on the site: Sometimes it’s as simple as opening up your browsers “Developer Tools” console and looking for XHR (XML HTTP ...

Another popular application of classification techniques is on texmining (see e.g. an old post on French president speaches). Consider the following example,  inspired by Nobert Ryciak’s post, with 12 wikipedia pages, on various topics, __ library(tm) __ library(stringi) __ library(proxy) __ titles = c("Boosting_(machine_learning)", + "Random_forest", + "K-nearest_neighbors_...

For the data scienec course of tomorrow, I just wanted to post some functions to illustrate cluster analysis. Consider the dataset of the French 2012 elections __ elections2012=read.table( "http://freakonometrics.free.fr/elections_2012_T1.csv",sep=";",dec=",",header=TRUE) __ voix=which(substr(names( + elections2012),1,11)=="X..Voix.Exp") __ elections2012=elections2012[1:96,] __ X=...