Finding Earth II

May 2, 2012
By
Finding Earth II

By 2030, we will have found approximately 10,000 exoplanets. "If it is just us... seems like an awful waste of space." -- from the movie Contact (1997) based on the book Contact by Carl Sagan. By the year 2030, it's possible that over ten th...

Read more »

Computational Journalism Server – The Way Forward

May 2, 2012
By

As I’ve noted here, the Computational Journalism Server “wants to be a Platform-as-a-Service (PaaS) when it grows up.” In plotting the way forward to that goal, I’ve looked at three options: Remain on openSUSE / SUSE Studio and ...

Read more »

Speeding up R with Intel’s Math Kernel Library (MKL)

May 2, 2012
By

I did some comparisons of the generic BLAS with Intel's MKL (both sequential and parallel) on a Dell PowerEdge 610 server with dual hyperthreading 6-core 3.06GHz Xeon X5675 processors.  Here are the results from an R benchmarking script (Normal R indicates the generic BLAS,  sMKL is the sequential (single core Intel MKL, and pMKL is the parallel Intel MKL using...

Read more »

Speeding up R with Intel’s Math Kernel Library (MKL)

May 2, 2012
By
Speeding up R with Intel’s Math Kernel Library (MKL)

I did some comparisons of the generic BLAS with Intel's MKL (both sequential and parallel) on a Dell PowerEdge 610 server with dual hyperthreading 6-core 3.06GHz Xeon X5675 processors.  Here are the results from an R benchmarking script (Normal R ...

Read more »

2nd round of call for chapter proposals for book Data Mining Applications with R: due by 31 May

May 2, 2012
By
2nd round of call for chapter proposals for book Data Mining Applications with R: due by 31 May

2nd CALL FOR CHAPTERS: proposals due by 31 May 2012 Data Mining Applications with R A book to be published by Elsevier http://www.RDataMining.com/books/book2 Introduction —————— R is one of the most widely used data mining tools in scientific and business … Continue reading →

Read more »

Measuring time series characteristics

May 2, 2012
By
Measuring time series characteristics

A few years ago, I was working on a project where we measured various characteristics of a time series and used the information to determine what forecasting method to apply or how to cluster the time series into meaningful groups. The two main papers to come out of that project were: Wang, Smith and Hyndman (2006) Characteristic-​​based clustering for...

Read more »

Next Kölner R User Meeting: 6 July 2012

May 1, 2012
By
Next Kölner R User Meeting: 6 July 2012

The next Cologne R user group meeting is scheduled for 6 July 2012. All details are available on the new KölnRUG Meetup site. Please sign up if you would like to come along, and notice that there is also pub poll for the after "work" drinks. Notes fr...

Read more »

A gallery view for Craigslist

May 1, 2012
By
A gallery view for Craigslist

As much as I love Craigslist, I sometimes find the interface a bit limited. My biggest wish? That there was an option for showing the search results as an image gallery, like eBay has. This could prove quite useful for browsing things like antiques,...

Read more »

Mining for relations between nominal variables

May 1, 2012
By
Mining for relations between nominal variables

The task today was to find what variables had significant relations with an important grouping variable in the big dataset I’ve been working with lately.  The grouping variable has 3 levels, and represents different behaviours of interest.  At first I … Continue reading →

Read more »

Playing with knitr: Create Report with Dynamic List

May 1, 2012
By
Playing with knitr: Create Report with Dynamic List

Here is a little toy example using knitr, LaTeX/MiKTeX and Google Docs.Say you had a list on Google Docs (say a list of attendants) and you want to print a report with it..Then see this example using this Rnw-file and the output...make the tex-file wit...

Read more »

Google BigQuery and the Github Data Challenge

May 1, 2012
By

Github has made data on its code repositories, developer updates, forks etc. from the public GitHub timeline available for analysis, and is offering prizes for the most interesting visualization of the data. Sounds like a great challenge for R programmers! The R language is currently the 26th most popular on GitHub (up from #29 in December), and it would...

Read more »

New R User Group in Cologne, Germany

May 1, 2012
By

The latest local R user group to join the fold is the Köln R User Group, now the sixth R user group in Germany. Their first group meeting will be on July 6, with presentations on ANOVA, ggplot2 graphics in Deducer, and writing R code with Emacs's Org-mode. If you're in the Cologne area, this would be a great...

Read more »

NSF BIGDATA webinar

May 1, 2012
By

If you're doing any kind of big data analysis - genomics, transcriptomics, proteomics, bioinformatics - then unless you've been on vacation the last few weeks you've no doubt heard about the NSF/NIH BIGDATA  Initiative (here's the NSF solicitation...

Read more »

Quick Tip: Replace Values in Dataframe on Condition with Random Numbers

May 1, 2012
By

This one took me some time - though, in fact it is plain simple:> options(scipen=999)> (my_df X1 X2 X3 X4 X5 X6 X7 X8 X9 X101 0 0 1 0 1 1 1 1 0 12 0 0 1 ...

Read more »

What does this package look like?

May 1, 2012
By
What does this package look like?

In this post, I give a very simple trick to understand the way a package is organized, which functions are included in and how these functions depend from each others. The idea has been brought by one of my student, Soraya, who is currently working in a very hostile environment, surrounded by true geeks. However,

Read more »

A Warning About warning()

May 1, 2012
By

Avoid R’s warning feature. This is particularly important if you use R in production; when you regularly run R scripts as part of your business process. This is also important if you author R packages. Don’t issue warnings in your own co...

Read more »

Monitoring some statistics with "R"

May 1, 2012
By
Monitoring some statistics with "R"

I´ve been practicing after reading a couple of tutorials:R: A self-learn tutorialProgramming in Rto create a basic function  to monitor some  basic statistics as RMSEP, Bias, SEP, Correlation and RSQ. I´ve been doing this with other so...

Read more »

How to Make HTML5 Slides with knitr

May 1, 2012
By
How to Make HTML5 Slides with knitr

One week ago I made an early announcement about the markdown support in the knitr package and RStudio, and now the version 0.5 of knitr is on CRAN, so I'm back to show you how I made the HTML5 slides. For those who are not familiar with markdown, you m...

Read more »

Volatility Position Sizing to improve Risk Adjusted Performance

April 30, 2012
By
Volatility Position Sizing to improve Risk Adjusted Performance

Today I want to show how to use Volatility Position Sizing to improve strategy’s Risk Adjusted Performance. I will use the Average True Range (ATR) as a measure of Volatility and will increase allocation during low Volatility periods and will decrease allocation during high Volatility periods. Following are two good references that explain these strategy

Read more »

Simple Moving Average Strategy with a Volatility Filter: Follow-Up Part 2

April 30, 2012
By
Simple Moving Average Strategy with a Volatility Filter: Follow-Up Part 2

In the Follow-Up Part 1, I explored some of the functions in the quantstrat package that allowed us to drill down trade by trade to explain the difference in performance of the two strategies. By doing this, I found that my choice of a volatility measure may not have been the best choice. Although the … Continue reading...

Read more »

Information Age: graduates driving industry adoption of R

April 30, 2012
By

Information Age recently published a feature article devoted to the R language, "Putting the R in analytics". Says author Pete Swabey: Already popular in universities, there are signs that R is finding increasing adoption in the enterprise. This promises to lower the barriers of entry for advanced analytics, and may accelerate the mathemitisation of business management. The article includes...

Read more »

French Global Factors

April 30, 2012
By
French Global Factors

I have said it already in multiple posts, but Kenneth French’s data library is one of the most generous and powerful contributions to the financial community.  To build on Systematic Investor’s series on factors, I thought I should run some ba...

Read more »

Bayesian ANOVA for sensory panel profiling data

April 30, 2012
By
Bayesian ANOVA for sensory panel profiling data

In this post it is examined if it is possible to use Bayesian methods and specifically JAGS to analyze sensory profiling data. The aim is not to obtain different results, but rather to confirm that the results are fairly similar. The data used is the c...

Read more »

Example 9.29: the perils of for loops

April 30, 2012
By
Example 9.29: the perils of for loops

A recent exchange on the R-sig-teaching list featured a discussion of how best to teach new students R. The initial post included an exercise to write a function, that given a n, will draw n rows of a triangle made up of "*", noting that for a beginner, this may require two for loops. For example,...

Read more »

Teaching code, production code, benchmarks and new languages

April 30, 2012
By
Teaching code, production code, benchmarks and new languages

I’m a bit obsessive with words. May be I should have used learning in the title, rather than teaching code. Or perhaps remembering code. You know? Code where one actually has very clear idea of what is going on; for … Continue reading →

Read more »

Cross-sectional skewness and kurtosis: stocks and portfolios

April 30, 2012
By
Cross-sectional skewness and kurtosis: stocks and portfolios

Not quite expected behavior of skewness and kurtosis. The question In each time period the returns of a universe of stocks will have some distribution — distributions as displayed in “Replacing market indices” and Figure 1. Figure 1: A cross-sectional distribution of simple returns of stocks. In particular they will have values for skewness and … Continue reading...

Read more »

Incompetence borne of excessive cleverness

April 29, 2012
By

I have just got back from the 24 hour Data Science Global Hackathon; I was an on-site participant at Hub Westminster in London (thanks to Carlos and his team for doing such a great job looking after us all {around 50 turned up from the 100 who registered; the percentage was similar in other cities

Read more »

The Need for paste2 (part II)

April 29, 2012
By
The Need for paste2 (part II)

This is Part II of a multi part blog on the paste2 function… In my first post on the paste2 function I promised a proof of a few practical uses.  The first example I have comes from psychometrics and comes out of … Continue reading →

Read more »

The R-Podcast Episode 6: Importing Data from External Sources

April 29, 2012
By

In this episode: Listener feedback and importing data from external sources into R. We dive into the basics of importing delimited text files using read.table and its varients. We also discuss recommendations for importing MS Excel spreadsheet files, relational databases such as MySQL, data from HTML tables, and files produced by other statistical computing packages.

Read more »