Monthly Archives: June 2013

Income Distribution in London

June 7, 2013
By
Income Distribution in London

Inspired by the Institute of Fiscal Studies' "Where do you fit in" application, where people can find out their position in the UK's income distribution, I wanted to find out how the picture in London looks like. Quite different. If you are in a very high percentile nationwide, high incomes of mainly financial sector employees in London...

Read more »

Symmetric set differences in R

June 7, 2013
By

My .Rprofile contains a collection of convenience functions and function abbreviations. These are either functions I use dozens of times a day and prefer not to type in full:## my abbreviation of head() h Or problems that I'd rather figure out once, and only once: ## example: ## between( 1:10, 5.5, 6.5 ) between = low & x low & x...

Read more »

Comrades Marathon Attrition Rate

June 7, 2013
By
Comrades Marathon Attrition Rate

It is a bit of a mission to get the complete data set for this year’s Comrades Marathon. The full results are easily accessible, but come as an HTML file. Embedded in this file are links to the splits for individual athletes. So with a bit of scripting wizardry it is also possible to download

Read more »

Creating Catch Data from Individual Length Measurements

June 6, 2013
By
Creating Catch Data from Individual Length Measurements

This example has been updated in this post. I came across a “problem” today where I needed to create catch data for individual nets from length measurements made on individual fish in those nets.  In other words, I had data … Continue reading →

Read more »

Data Class Conversion

June 6, 2013
By

Data in R can be converted from one class to the other. The function is prefixed with as. then followed by the name of the data class that we wish to convert to. Data class in R are the following:numeric - as.numericvector - as.vectorcharacter - as.cha...

Read more »

How likely is the NSA PRISM program to catch a terrorist?

June 6, 2013
By
How likely is the NSA PRISM program to catch a terrorist?

Recent revelations about PRISM, the NSA’s massive program of surveillance of civilian communications have caused quite a stir. And rightfully so, as it appears that the agency has been granted warrantless direct access to just about any form of digital communication engaged in by American citizens, and that their access to such data has been

Read more »

Feature Selection 3 – Swarm Mentality

June 6, 2013
By
Feature Selection 3 – Swarm Mentality

"Bees don't swarm in a mango grove for nothing. Where can you see a wisp of smoke without a fire?" - Hla Stavhana In the last two posts, genetic algorithms were used as feature wrappers to search for more effective subsets of predictors. Here, I will do the same with another type of search algorithm: particle swarm optimization....

Read more »

Intro to Parallel Random Number Generation with RevoScaleR

June 6, 2013
By
Intro to Parallel Random Number Generation with RevoScaleR

by Joseph Rickert Random number generation is fundamental to doing computational statistics. As you might expect, R is very rich in random number resources. The R base code provides several high quality random number generators including: Wichmann-Hill, Marsaglia-Multicarry, Super-Duper, Mersenne-Twister, Knuth-TAOCP-2002 and L’Ecuyer-CMRG. (See Random for details.) And, there are at least three packages, rspring, rlecuyer, and rstream for...

Read more »

Box-plot with R – Tutorial

June 6, 2013
By
Box-plot with R – Tutorial

Uncertain Demand Forecasting and Inventory Optimizing for Short-life-cycle Products

June 6, 2013
By

For short-life-cycle products such as newspapers and fashion, it is important to match the supply with the demand. However, sometimes we order too little from supplier and sometimes we order too much due to the uncertain demand. We would lose sales and customers would be unsatisfied if ordering too little or we would let the

Read more »