art brut

November 12, 2011
By
art brut

Filed under: pictures, R Tagged: Le Monde, mathematical puzzle, R

Read more »

Visualizing Likert Items

November 11, 2011
By
Visualizing Likert Items

I have become quite a big fan of graphics that combine the features of traditional figures (e.g. bar charts, histograms, etc.) with tables. That is, the combination of numerical results with a visual representation has been quite useful for exploring descriptive statistics. I have wrapped two of my favorites (build around ggplot2) and included them as part

Read more »

Web Scraping Google+ via XPath

November 11, 2011
By
Web Scraping Google+ via XPath

Google+ just opened up to allow brands, groups, and organizations to create their very own public Pages on the site. This didn’t bother me to much but I’ve been hearing a lot about Google+ lately so figured it might be fun to set up an XPath scraper to extract information from each post of a status

Read more »

Harmonic means, reciprocals, and ratios of random variables

Harmonic means, reciprocals, and ratios of random variables

In my last few posts, I have considered “long-tailed” distributions whose probability density decays much more slowly than standard distributions like the Gaussian.  For these slowly-decaying distributions, the harmonic mean often turns out to be a much better (i.e., less variable) characterization than the arithmetic mean, which is generally not even well-defined theoretically for these distributions.  Since the harmonic...

Read more »

Propagation of error

November 11, 2011
By
Propagation of error

     At the onset, this was strictly an excercise of my own curiosity and I didn't imagine writing this down in any form at all. As someone who has done some modelling work in the past, I'm embarrassed to say that I had never fully grasped how one can gauge the error of a...

Read more »

Pre-computing a trading plan in parallel

November 11, 2011
By
Pre-computing a trading plan in parallel

R version 2.14 introduced a new package, called parallel. This new package combines the functionality from two previous packages: snow and multicore. Since I was using multicore to parallelise my computations, I had to migrate to the new package and decided to publish some code. Often trading strategies are tested using the daily closing price

Read more »

A chart for marathoners

November 11, 2011
By
A chart for marathoners

Here's a cool application of calendar heat maps: runner Andy used R to catalogue his daily running mileage over the last 2+ years: There are lots of ways to chart data like this (a simple time-series chart, for example), but sometimes looking at data in new ways offers fresh perspectives. For example, Andy notes: "Apparently I missed running on...

Read more »

The Marriage of Hadoop and R: Revolution Analytics at Hadoop World

November 11, 2011
By
The Marriage of Hadoop and R: Revolution Analytics at Hadoop World

Revolution Analytics CTO David Champagne visited Hadoop World 2011 this week, and delivered a presentation on "The Powerful Marriage of R and Hadoop" to a standing-room-only crowd of R and Hadoop enthusiasts. I've included David's slides below: The talk also generated praise on Twitter, for example: David was also interviewed by The Cube during the conference. In the video...

Read more »

Train neural network in R, predict in SAS

November 11, 2011
By
Train neural network in R, predict in SAS

This R code fits an artificial neural network in R and generates Base SAS code, so new records can be scored entirely in Base SAS. This is intended to be a simple, elegant, fast solution. You don’t need SAS Enterprise … Continue reading →

Read more »

RStudio: a cross-platform IDE for R

November 11, 2011
By
RStudio: a cross-platform IDE for R

Which text editor do you use? Once in a while this question pops up on the R-help mailing list. Up until recently I used the KDE text editor Kate under Linux. Recently, I came across a new text editor for… See more ›

Read more »

Resampling and Shrinkage : Solutions to Instability of mean-variance efficient portfolios

November 11, 2011
By
Resampling and Shrinkage : Solutions to Instability of mean-variance efficient portfolios

Small changes in the input assumptions often lead to very different efficient portfolios constructed with mean-variance optimization. I will discuss Resampling and Covariance Shrinkage Estimator – two common techniques to make portfolios in the mean-variance efficient frontier more diversified and immune to small changes in the input assumptions. Resampling was introduced by Michaud in Efficient

Read more »

What you wish you knew before you started a PhD

November 11, 2011
By
What you wish you knew before you started a PhD

I asked my research group recently what they wished they had learned before they started work on a PhD. Here are some of the responses. More mathematics. Particular topics they named included real analysis, functional analysis, measure theory, algebra, linear algebra. That would have been my response also. I still wish I knew more mathematics than

Read more »

Another look at autocorrelation in the S&P 500

November 11, 2011
By
Another look at autocorrelation in the S&P 500

Casting doubt on the possibility of mean reversion in the S&P 500 lately. Previously A look at volatility estimates in “The mystery of volatility estimates from daily versus monthly returns” led to considering the possibility of autocorrelation in the returns.  I estimated an AR(1) model through time and added a naive confidence interval to the … Continue reading...

Read more »

Surviving a binomial mixed model

November 11, 2011
By
Surviving a binomial mixed model

A few years ago we had this really cool idea: we had to establish a trial to understand wood quality in context. Sort of following the saying “we don’t know who discovered water, but we are sure that it wasn’t … Continue reading →

Read more »

Visualizing Likert Items

November 11, 2011
By
Visualizing Likert Items

I have become quite a big fan of graphics that combine the features of traditional figures (e.g. bar charts, histograms, etc.) with tables. That is, the combination of numerical results with a visual representation has been quite useful for exploring...

Read more »

Visualizing Likert Items

November 11, 2011
By
Visualizing Likert Items

I have become quite a big fan of graphics that combine the features of traditional figures (e.g. bar charts, histograms, etc.) with tables. That is, the combination of numerical results with a visual representation has been quite useful for exploring...

Read more »

Plotting implicit functions in R

November 11, 2011
By
Plotting implicit functions in R

So in prepping for my latest manuscript on population dynamics I have been creating all the necessary figures.  One of them I considered was a 2-d surface plot of a modified Ricker equation showing the transitions from extinction stability, and stability to limit cycles.  Inconveniently though the only way to do this is with an implicit function.  Since becoming...

Read more »

What 5,728.986 miles look like…

November 10, 2011
By
What 5,728.986 miles look like…

Time Series as calendar heat maps + All of my running data since April 1, 2009 = Generated by the following code: #Sample Code based on example program at: source(file = "calendarHeat.R") run<- read.csv("log.csv", header = TRUE, sep=",") sum(run$Distance) date <- c() for (i in 1: dim(run)){ if(run$DistanceUnit== 'Kilometer'){ miles <- c(miles,run$Distance * 0.62) }

Read more »

Managing a Local R Repository

November 10, 2011
By

I will be teaching a workshop on R and LaTeX at NEAIR in just under a month. One of the issues I will encounter is a lack of Internet access. I also work with restricted data from NCES which requires the computer to be secured including no network access. As such, I need to manage software from removable

Read more »

Applying multiple functions to data frame

November 10, 2011
By
Applying multiple functions to data frame

A very typical task in data analysis is calculation of summary statistics for each variable in data frame. Standard lapply or sapply functions work very nice for this but operate only on single function. The problem is that I o...

Read more »

PostGIS Quickie

November 10, 2011
By

Today I needed to cut out a rectangle of geologic data from a state-wide map in an AEA coordinate system, using a bounding box from a UTM zone 10 region, with the output saved in UTM zone 10 coordinates. PostGIS makes this type of operation very simple...

Read more »

Web Scraping Yahoo Search Page via XPath

November 10, 2011
By
Web Scraping Yahoo Search Page via XPath

Seeing as I’m on a bit of an XPath kick as of late, I figured I’d continue on scraping search results but this time from Yahoo.com Rolling my own version of xpathSApply to handle NULL elements seems to have done the trick and so far it’s been relatively easy to do the scraping. I’ve created

Read more »

Changing world, Changing JGB term structure

November 10, 2011
By
Changing world, Changing JGB term structure

Writing the article "How much does "Beta" change depending on time?", I learned how to create an animation by using R language. Then, I would like to continue do that in this article.In this article, I visualize time series of JGB term struct...

Read more »

The mysterious case of the misbehaving writeLines() (or: a cat saves the day)

November 10, 2011
By
The mysterious case of the misbehaving writeLines() (or: a cat saves the day)

Dear readers and R experts, I submit to you a mysterious R quirk which has been baffling me for the best part of a week. I found a work-around but I’d love it if someone could explain this strangest of … Continue reading →

Read more »

An Image Crossfader Function

November 10, 2011
By
An Image Crossfader Function

Some project offspin, the jpgfader-function (the jpgfader-function in funny use can be viewed HERE):Read more »

Read more »

In case you missed it: October Roundup

November 10, 2011
By

In case you missed them, here are some articles from October of particular interest to R users. The creator of the ggplot2 package, Hadley Wickham, shares details on some forthcoming big-data graphics functions (based on research sponsored by Revolution Analytics). A list of several dozen free data sources that can easily be imported into R. Bob Muenchen gave a...

Read more »

Code optimization, an Rcpp solution

November 10, 2011
By

Tony Breyal woke up an old code optimization problem in this blog post, so I figured it was time for an Rcpp based solution This solutions moves down Henrik Bengtsson's idea (which was at the basis of attempt 10) down to C++. The idea was to call sprintf less than the other solutions to generate the strings...

Read more »

Expected Salary by Major

November 10, 2011
By
Expected Salary by Major

In this recent editorial about the Occupy Wall Street movement, Richard Kim profiles a protestor that despite having a master’s degree can’t find a job. This particular protestor quit his job as a school teacher three years ago and took out a $...

Read more »

Facebook Graph API Explorer with R

November 10, 2011
By
Facebook Graph API Explorer with R

I wanted to play around with the Facebook Graph API  using the Graph API Explorer page as a coding exercise. This facility allows one to use the API with a temporary authorisation token. Now, I don’t know how to make an R package for the proper API where you have to register for an API key and

Read more »