I have finally taken the plunge and created my first R package! As frequent readers will know, I often sing the praises of infochimps, a startup out of Austin, TX attempting to be the world’s data clearinghouse. While infochimps is an excellent resource for data sets, they also provide ... [Read more...]

HugeFileLoader = function(path, sep = "\t", skip = 0, header = T, nrows = 10){### counts the number of lines using shell wc command, and converts the output to numericline.count = paste("wc -l ", path, sep = "")row.count = as.numeric(strsplit(system(li... [Read more...]

Welcome to my new category: ShortCut! Here I'll shortly explain some smart features, unknown extensions or uncommon pathways of going for gold. Today it's about the Gnu R tool locator. [Read more...]

Aquamacs in Mac OS X and Emacs in Linux/unix can be used to edit remote (and local) R code and submit pieces of code to a remote R session. For this to work you need to install ess for emacs (Aquamacs comes with ess by default now, I don't ... [Read more...]

[Read more...]

Nathan Yau, of the excellent FlowingData blog, recently asked on his Twitter stream: I wonder if there’s a market for premium R packages, like there is for say, @wordpress themes and plugins There are some great packages available for R, all of which are currently free. I think it ... [Read more...]

The United States has recently introduced millimeter wave and backscatter x-ray scanners to the security screening process in many airports, prompting a backlash in some quarters. Much of the opposition is centered around the invasion of privacy: the scanners generate an image of the traveller's naked body. There are also ... [Read more...]

Spending the day churning through large data set or doing some heavy-duty number crunching? What is one to do while the computer is running in overdrive? We’ll, for one, you could get a steaming cup of joe and write a … Continue reading → [Read more...]

A quick reminder that two competitions based around data analysis, both very suited to R, are currently underway. First, there's still plenty of time to enter the competition to predict popular R packages, announced by the The Dataists and hosted at Kaggle. According to organizer Drew Conway, the competition has ... [Read more...]

RapidMiner + R is an advanced tool that can be used to analyze trading strategies, In order to check its power I made a simple example using an algorithm based on a support vector machine for predicting the next day's price and based on it I generated ... [Read more...]

I recently stumbled upon an old paper of a presentation I gave at the Interface conference in 1998, entitled “JAVA – the next Generation of Statistical Computing?”: It is very interesting to compare the things I envisioned 12 years ago and what actually came true. Here are some topics: Did Java change a ... [Read more...]

Please note - I’ve spotted a problem with the approach taken in this post – it seems to underestimate power in certain circumstances. I’ll post again with a correction or a more full explanation when I’ve sorted it. So, I posted an answer on cross validation regarding logistic ... [Read more...]

The R package dcemriS4 is a collection of functions, with examples and documentation, that allows one to perform voxel-wise quantitative analysis of dynamic contrast-enhanced MRI (DCE-MRI) or diffusion-weighted imaging (DWI) data.  The primary... [Read more...]

The class started yesterday with a small but focussed and responsive audience! Given the background of the students, and in particular their clear proficiency in R!, I switched between the original slides of Introducing Monte Carlo Methods with R and those of my Monte Carlo Statistical Methods: course, updated by ... [Read more...]

Revolution Analytics is growing, and we're looking for some skilled R Hackers to work in our pre-Sales team. A big part of our task is showing companies how R is such a great tool for modern data analysis (especially compared to those older tools with 3- or 4-letter acronyms). So ... [Read more...]

Roll [1984] proposed a model for the bid-ask spread that was based on first-order serial correlation.  His empirical tests were based on daily and weekly frequency equity data, and based on the results he concluded there were informational inefficiencies (or that there was very short term non-stationarity in expected returns).More ... [Read more...]

A few months ago I showed you how to syntax-highlight R code using Github Gists for displaying R code on your blog or other online medium. The idea's really simple if you use blogger - head over to gist.github.com, paste in your R code, create a public "gist", ... [Read more...]

Google Earth Access Trends: Daily Requests read more [Read more...]

By guest blogger Joseph Rickert. I was very happy to be a part of the ACM Data Mining camp held last Saturday (November 13th) at eBay. It was a big day for discussing hot topics in data mining, Mahout, parallel SVMs etc, and also a pretty big day for R. ... [Read more...]

A few weeks ago I wrote about ways to compare major-party returns in US House elections. I experimented with several visualizations, none as useful as the seats-votes curve. A traditional seats-votes cure measures average party performance against individual US House results. Our simplified curve uses a density plot to measure ... [Read more...]