## Paired sample t-test in R

September 28, 2011
By

Let’s walk through using R and Student’s t-test to compare paired sample data. The book Statistics: The Exploration & Analysis of Data (6th edition, p505) presents the longitudinal study “Bone mass is recovered from lactation to postweaning in adolescent mothers … Continue reading →

## ttrTests This is a Test–Test 1 and Test 2

September 28, 2011
By

Just to remind everyone, THIS IS NOT INVESTMENT ADVICE AND ANY ACTIONS TAKEN BASED ON THIS DISCUSSION WILL PROBABLY RESULT IN SIGNIFICANT LOSSES. We had fun with the ttrTests package in two previous posts ttrTests: Its Great Thesis and Incredible Poten...

## The R Graph Gallery goes social

September 28, 2011
By

The R Graph Gallery, the website from Romain François that showcases hundreds of examples of data visualization with R, has new social features. Now, when you find a graph or chart you find appealing or useful, you can "Like" it on Facebook or "+1" it on Google+. This should be a great way of highlighting the best charts and...

## Is the “Long Tail” a Useless Concept?

In response to my last post, “The Long Tail of the Pareto Distribution,” Neil Gunther had the following comment:            “Unfortunately, you've fallen into the trap of using the ‘long tail’ misnomer. If you think about it, it can't possibly be the length of the tail that sets distributions like Pareto and Zipf apart; even the negative exponential and Gaussian...

## Data Science: a literature review

September 28, 2011
By

Just what is Data Science, anyway? Here's one take: Ever since the term "Data Scientist" was coined by DJ Patil and Jeff Hammerbacker in 2009, there's been a vigorous debate on what the team actually means. More than 80% of statisticians consider themselves data scientists, but Data Science is more than just Statistics. (My own take is that Data...

## Polyploidy in sugarcane

September 28, 2011
By

While reading UseR conference abstracts I came across this sentence: "Sugarcane is polypoid, i.e., has 8 to 14 copies of every chromosome, with individual alleles in varying numbers." Vau! This generates really complex genotype system. Say we have biallelic gene with alleles being A and B. In diploids the possible genotypes are AA, AB, and BB. Given the...

## Bessel integral

September 28, 2011
By
$Bessel integral$

Pierre Pudlo and I worked this morning on a distribution related to philogenic trees and got stuck on the following Bessel integral where In is the modified Bessel function of the first kind. We could not find better than formula 6.611(4) in Gradshteyn and Ryzhik. which is for a=0… Anyone in for a closed form

## Using transparency for data count intuition

September 27, 2011
By

This is an illustration of representing point count in a graphic using transparency. This is easy to do in ggplot2 if you use one of the barchart type of geoms.  However I think there are other situations where it would be useful to apply aesthetics based on point count. Since Hadley did a lot of

## World Tourism Day, and Google Public Data Explore

September 27, 2011
By

Today is the World Tourism Day! So let’s speak about some tourism related datasets – and others. Among other nice functions, Google offers a Public Data Explore in a beta version which provides a collection of datasets from OECD, IMF, Eurostat, … Continue reading →

## Five new local R user groups

September 27, 2011
By

Looks like there's been a lot of activity in the R user community in the Northern hemisphere now that the summer break is over. I've just added several new groups to the Local R User Group Directory: Tokyo, Japan: The Tokyo.R R study group has already had 17 meetings, but has just been added to the directory. Shanghai/East China:...

## Tikz Introduction

September 27, 2011
By

The pgf drawing package for LaTeX provides facilities for drawing simple of complicated pictures within a LaTeX document. There are many options available within the package and in this post we consider some of the basics to get up and running. Fast Tube by Casper As with all LaTeX documents we need to select a

## Basic line chart with ggplot2

September 27, 2011
By

ggplot2 is a package for R which easily draws plots that are easier on the eyes than R’s built-in plotting functions, though the grammar is different than what is commonly used in R. This code demonstrates how to prepare a … Continue reading →

## Ghastly R code

September 27, 2011
By

My R package, R/qtl, contains about 33k lines of R code (and 21k lines of C code). Some of it is quite good; some of it is terrible. Here’s another example of the terrible. I’ve long needed to revise the function scantwo, for performing a two-dimensional genome scan for pairs of loci. I was looking

## Project Euler: problem 6

September 27, 2011
By

The sum of the squares of the first ten natural numbers is,12 + 22 + ... + 102 = 385The square of the sum of the first ten natural numbers is,(1 + 2 + ... + 10)2 = 552 = 3025Hence the difference between the sum of the squares o...

## Example 9.7: New stuff in SAS 9.3– Frailty models

September 27, 2011
By

Shared frailty models are a way of allowing correlated observations into proportional hazards models. Briefly, instead of l_i(t) = l_0(t)e^(x_iB), we allow l_ij(t) = l_0(t)e^(x_ijB + g_i), where observations j are in clusters i, g_i is typically norma...

## Obama recruiting analysts and R is one preferred skill

September 27, 2011
By

Barack Obama is recruiting analysts for his 2012 re-election campaign. It is to analyze the campaign’s data to guide election strategy and develop quantitative, actionable insights that drive decision-making. R is mentioned as one of the tools to use. Analytics … Continue reading →

## Time series equivalence of brains and markets

September 27, 2011
By

fMRI data from 90 locations in the brain look somewhat like daily closing prices on 116 stocks if you squint just right. Marginal Revolution was nice enough to point to “Topological isomorphisms of human brain and financial market networks”. I’ve only just glanced through the paper.  I find it interesting, but I’m fairly skeptical.  The … Continue reading...

## Hipster programming languages

September 26, 2011
By

If you look at the programming languages that are popular these days, a few patterns emerge. I'm not talking about languages that have the most hits on the job sites. I'm talking about what the cool kids are coding in - the folks that hang out on hacke...

## Revolution Analytics partners with Cloudera

September 26, 2011
By

Revolution Analytics today announced that it has partnered with Cloudera, the leader in Apache Hadoop-based software and services, to make big-data analytics with Hadoop and R available to Revolution R Enterprise users. As we announced earlier this month, we have created three open-source R packages which make it possible for R users to write map-reduce programs in the R...

## ttrTests: Its Great Thesis and Incredible Potential

September 26, 2011
By

I stumbled on the ttrTests R package as mentioned in my post ttrTests Experimentation.  I did not recognize its potential until I spent much more time absorbing the basis of the package—David St. John’s thesis Technical Analysis Based on Movin...

## workshop in Columbia [day 3]

September 26, 2011
By

Although this was only a half-day of talks, the third day of the workshop was equally thought-challenging and diverse.  (I managed to miss the ten first minutes by taking a Line 3 train to 125th street, having overlooked the earlier split from Line 1… Crossing south Harlem on a Sunday morning is a fairly mild

## Using Inkscape to Post-edit Labels in R Graphs

September 26, 2011
By

I discuss how to use Inkscape to easily shift around labels on graphs produced in R. Continue reading →

## Gamified

September 26, 2011
By

Barry Rowlingson gave an interesting talk at UseR 2011, “Why R-help must die!” He suggested the Q-and-A type sites Stack Overflow (on programming) and Cross Validated (on statistics), both part of Stack Exchange. An interesting feature of these sites is that, in addition to voting up and down on the questions and answers, one accrues

## Visualizing Sampling Distributions

September 25, 2011
By

Teacher: “How variable is your estimate of the mean?” Student: “Uhhh, it’s not. I took a sample and calculated the sample mean. I only have one number.” Teacher: “Yes, but what is the standard deviation of sample means?” Student: “What do you mean means, I only have the one friggin number.” Statisticians have a habit

## Accessing and plotting World Bank data with R

September 25, 2011
By

Over the past couple of days I played around with the data sets of the World Bank, and I have to admit that I am blown away by it. It is amazing, to see what is available on their web site. It is worth visiting their Data Visualisation Tools page. It i...

## rrdf 1.5: Accessing SMW SPARQL end points behind LDAP authentication

September 25, 2011
By

We are using a Semantic MediaWiki (SMW) for the Gold Compound selection task by the ToxBank in the SEURAT-1 cluster, funded by Colipa and the EC. I do stress that despite being funded by Colipa, they have no control over my research; they just co-...

## Arc Diagram and spatiotemporal data mining visualization

September 23, 2011
By

I won't spend too much time discussing this fascinating topic other than to say it relates very much to prior discussions about pattern discovery via visual data mining (see lexical dispersion plots for example).  I happened across an interesting ...

## Nice Species Distribution Maps with GBIF-Data in R

September 23, 2011
By

Here's an example of how to easily produce real nice distribution maps from GBIF-data in R with package maps... Read more »

## Data Visualization doesn’t need to be biased

September 23, 2011
By

At the FlowingData blog, data visualization commentator and Visualize This author Nathan Yau lists 5 misconceptions about visualization: Software does everything (Nathan notes "Personally, I use a lot of R and have a lot of fun in Illustrator", but uses a lot of other tools as well.) Visualization is for making data flashy The more information in a single...