Tracking execution paths

June 18, 2011
By
Tracking execution paths

Earlier this week, I was trying to figure out the path of execution through a big chunk of code. Once you reach a certain size of codebase, tracking which function gets called when can be tricky. My first thought for dealing with this was to add a message line at the start of each function

Read more »

A Brief Introduction to Mixture Distributions

A Brief Introduction to Mixture Distributions

Last time, I discussed some of the advantages and disadvantages of robust estimators like the median and the MADM scale estimator, noting that certain types of datasets – like the rainfall dataset discussed last time – can cause these estimators to fail spectacularly.  An extremely useful idea in working with datasets like this one is that of mixture distributions,...

Read more »

Exploring the Market with Hurst

June 17, 2011
By
Exploring the Market with Hurst

Randomly trudging through PerformanceAnalytics source code, I was intrigued by the Hurst Index calculation, which I discovered is more commonly called Hurst Exponent.  After quickly satisfying myself that I could actually do the rolling Hurst calculat...

Read more »

Raster, CMSAF and solaR

Raster, CMSAF and solaR

The Satellite Application Facility on Climate Monitoring (CMSAF) generates, archives and distributes widely recognised high-quality satellite-derived products and services relevant for climate monitoring in operational mode. The data is freely accesible here after a registration process. I have ask them for several files with monthly averages of global solar radiation over the Iberian Peninsula (download).

Read more »

Big-Data PCA: 50 years of stock data

June 17, 2011
By
Big-Data PCA: 50 years of stock data

In this post, Revolution engineer Sherry LaMonica shows us how to use the RevoScaleR big-data package in Revolution R Enterprise to do principal components analysis on 50 years of stock market data -- ed. Principal components analysis, or PCA, seeks to find a set of orthogonal axes such that the first axis, or first principal component, accounts for as...

Read more »

solaR 0.24 at CRAN

solaR 0.24 at CRAN

The version 0.24 of solaR is at CRAN and R-Forge. Some days before the 0.23 version was uploaded, but I had to make a quick fix to readMAPA: the url of http://www.mapa.es/siar has been changed to http://www.marm.es/siar. Moreover, this function has been renamed to readSIAR, although it is still available as readMAPA. Consequently the mode

Read more »

Engineering Data Analysis (with R and ggplot2) – a Google Tech Talk given by Hadley Wickham

June 17, 2011
By
Engineering Data Analysis (with R and ggplot2) – a Google Tech Talk given by Hadley Wickham

It appears that just days ago, Google Tech Talk released a new, one hour long, video of a presentation (from June 6, 2011) made by one of R’s community more influential contributors, Hadley Wickham. This seems to be one of the better talks to send a programmer friend who is interested in getting into R.

Read more »

serialize or turn a large parallel R job into smaller chunks for use with SGE

June 16, 2011
By

I use the snow package in R with OpenMPI and SGE quite often for my simulation studies; I’ve outlined how this can be done in the past. The ease of these methods make it so simple for me to just specify the maximum number of cores available all the time. However, unless you own your... Read more »

REITs for Everybody Now REITs for Nobody Part 2

June 16, 2011
By
REITs for Everybody Now REITs for Nobody Part 2

As a quick follow-up to my first REITs for Everybody Might Now Mean REITs for Nobody, I want to look at REITs and High Yield bonds, which also might simultaneously attract conservative yield buyers and speculative beta chasers.HYG (iShares High Yield) ...

Read more »

Where Ichiro Hits

June 16, 2011
By
Where Ichiro Hits

Google research scientist Peter Hauck used Weka and k-means cluster analysis to describe where Mariners right-fielder Ichiro favours hitting the baseball. He then used R to visualize the 6 clusters the k-means analysis identified: I sometimes find K-means clusting tough to explain as a statistical technique, but this makes for a great example: if you're a fielder facing Ichiro,...

Read more »

5000 R questions on stackoverflow.com

June 16, 2011
By

The R tag on stackoverflow.com hit a milestone yesterday: 5000 questions about the R language. (The 5000th question was about the fortunes package, incidentally -- thanks to Andrie de Vries for pointing this out on Twitter.) Stackoverflow.com continue...

Read more »

Fixed: Unable to plot a decent x-Axis in a time series plot using zoo

June 16, 2011
By

Here is the link to the original problem. Briefly, I was unable to plot a custom x-axis showing abbreviated months in a time series plot of a zoo object. In the plot.zoo() function set xaxt = "n" to suppress plotting … Continue reading →

Read more »

Character occurrence in passwords

June 16, 2011
By
Character occurrence in passwords

As everyone knows, it seems that Sony is taking a bit of a battering from hackers.  Thanks to Sony, numerous account and password details are now circulating on the internet. Recently, Troy Hunt carried out a brief analysis of the password structure. Here is a summary of his post: There were around 40,000 passwords, of which

Read more »

Market arrows

June 16, 2011
By
Market arrows

Graphs like Figure 1 are reasonably common.  But they are not reasonable. Figure 1: A (log) price series with an explicit guide line. Some have the prices on a logarithmic scale, which is an improvement on the raw prices. The problem with this sort of plot is that two particular data points are taken as … Continue reading...

Read more »

How to plot points, regression line and residuals

June 16, 2011
By
How to plot points, regression line and residuals

x y # plot scatterplot and the regression linemod1 plot(x, y, xlim=c(min(x)-5, max(x)+5), ylim=c(min(y)-10, max(y)+10))abline(mod1, lwd=2)# calculate residuals and predicted valuesres pre # plot distances between points and the regression linesegments(x, y, x, pre, col="red")# add labels (res values) to pointslibrary(calibrate)textxy(x, y, res, cx=0.7)

Read more »

RTextTools now 100% Java-free!

When we first wrote RTextTools, we opted to use RWeka for boosting and bagging algorithms for lack of a better alternative. We've discovered that this leads to all sorts of ugly rJava installation issues across platforms and prevents our users from getting started quickly. Recently, we've stumbled upon two excellent non-Java alternatives: LogitBoost in the

Read more »

Further Bernoulli factories

June 15, 2011
By
Further Bernoulli factories

Yesterday, Andrew Thomas and José Blanchet posted a note on the Bernouilli factory on arXiv. This short paper links with the recent paper of Flegal and Herbei I commented earlier. Considering the special target Thomas and Blanchet develop an elaborate scheme of cascading envelopes that converge to f from above. Their paper is very clear

Read more »

The Big Analytics Revolution starts with R

June 15, 2011
By

Thanks to everyone who attended our webinar The 'Big Analytics' Revolution Starts with R yesterday. If you missed the live session, you can download the presentation slides (PDF) and the 30-minute replay video (WMV) from the Revolution Analytics website. The presentation focuses on the isse of Big Data, and how businesses can use advanced analytics methods implemented in the...

Read more »

R: Analyisis of a Sport Event

June 15, 2011
By
R: Analyisis of a Sport Event

Inspired by a post by a

Read more »

Statistical Analysis of the LAC Degerloch Volkslauf 2010

June 15, 2011
By
Statistical Analysis of the LAC Degerloch Volkslauf 2010

Inspired by a post by a R-blogger my interest was piqued to examine the runs in my athletic club. Therefore, I started R and analysed he LAC Degerloch Volkslauf 2010; a 10km race near Stuttgart-Hoffeld. Next lines, I present this statistical examination. The data can be found at: data. Firstly, I converted the data

Read more »

Sachin Tendulkar’s longevity

June 15, 2011
By
Sachin Tendulkar’s longevity

There have been over 3300 cricketers who've played Test and One Day cricket. The youngest player was 14 years old Hasan Raza from Pakistan who played 5 ODIs and 2 Test matches at that age. The oldest player was 52 years old&nbs...

Read more »

sas7bdat database reader update

June 14, 2011
By

An earlier post (1216) introduced a compatibility study (i.e. reverse engineering) of the sas7bdat database file format. The code and documentation for this are here: http://github.com/biostatmatt/sas7bdat. I've recently restructured the code as an R package, and added some functionality. Look for the sas7bdat package on the CRAN. Also, the read.sas7bdat code has been ported to

Read more »

Embedding a time series with time delay in R — Part II

June 14, 2011
By
Embedding a time series with time delay in R — Part II

Some months ago, I posted a function that extended the base R function embed() to allow for time delay embedding. Today, David Gonzales alerted me to an inconsistency between embed() and Embed(). The example David used was where Embed() clearly … Continue reading →

Read more »

Importing Nanotoxicity Data with SPARQL into R for analysis

June 14, 2011
By

Not so long ago I wrote about mporting RDF input in R for analysis. I am collecting nanotoxicology data in a Semantic MediaWiki with the RDFIO extension installed (by Samuel), allowing me to SPARQL that data directly from R. There is nothing much structural to visualize at this moment, so I'm skipping the Bioclipse...

Read more »

Hot Job in IT: Data Science

June 14, 2011
By

CIO Magazine today has an article on the "6 Hottest New Jobs in IT" in which features Data Science and R at #2: "There's now an intellectual consensus in business that the only way to run an enterprise is to use analytics with data scientists to find opportunities," says Norman Nie, CEO of Revolution Analytics, which produces the first...

Read more »

Wilcoxon Champagne test

June 14, 2011
By
Wilcoxon Champagne test

As an appetizer for Paris triathlon, Jérôme and I ran as a team last week-end an adventure racing in Champagne region (it mainly consists in running, cycling, canoeing, with a flavor of orienteering, and Champagne is kept for the end). It was organized by Ecole Polytechnique students who, for the first time, divided Saturday’s legs

Read more »

REIT Momentum in Quantstrat

June 14, 2011
By
REIT Momentum in Quantstrat

I took a short break from quantstrat to do some REIT analysis REITs for Everybody Might Now Mean REITs for Nobody.  Now let’s link the two by incorporating The Aleph Blog momentum bucket strategy in quantstrat. From TimelyPortfolio In ...

Read more »

RStudio Beta 3 (v0.94)

June 14, 2011
By
RStudio Beta 3 (v0.94)

RStudio Beta 3 (v0.94) is available for download today. The goal for this release was to refine and improve our core features based on the feedback we’ve gotten on our first two betas. Highlights of the new release include: Source editor enhancements — New editor features include brace/paren/quote matching, more intelligent cursor placement after newlines, function

Read more »

Boxplots without boxes

June 14, 2011
By

Let’s say you have several categories with multiple data points each that you would like to plot as individual points. Even if you have only a single point, the R graphics package will plot a line (without a box for … Continue reading →

Read more »