The performance cost of a for-loop, and some alternatives

August 21, 2011
By

I’ve recently been spending a lot of time running various simulations in R. Because I often use snow to perform simulations across several computers/cores, results typically come back in the form of a list object. Summarizing the results from a list … Continue reading →

Read more »

tty Connection + sas7bdat: useR! 2011 Presentation Slides

August 21, 2011
By
tty Connection + sas7bdat: useR! 2011 Presentation Slides

Experimenting with a tty Connection for R I presented twice at this years useR!. The first was a regular talk on the tty connection patch for R. The talk went smoothly, despite a live demonstration using the DLP-232PC data acquisition module (datasheet). The slides for this presentation are here: shotwell-tty-useR-2011.pdf The image above is a

Read more »

Prime testing function in R

August 20, 2011
By
Prime testing function in R

I was hoping to begin tinkering a bit with the multicore package in R beyond some extremely trivial examples.  Thanks to a combination of R’s dumb quirkiness (for example, being worthless on loops), my poor planning, and general bad programming, my Saturday afternoon tinkering project is ultimately worthless in fulfilling that purpose. I was really

Read more »

useR! Conference 2011 highlights

August 20, 2011
By
useR! Conference 2011 highlights

I was at the useR! Conference at The University of Warwick in Coventry, UK, last week. My goal in going was to learn the latest things regarding (simple) dynamic graphics, (simple) web-based apps, parallel computing, and memory management (dealing with big data sets). I got just what I was hoping for and more. There are

Read more »

Statistical Analysis Functions in R

August 20, 2011
By

Lately, I've been using statistical tests on a daily basis. I've noticed that I have to format my data the same way in order to get it into R (tab-delimited flat file essentially). Every other change in order to prep that data structure for any sort of...

Read more »

When are averages useless?

When are averages useless?

Of all possible single-number characterizations of a data sequence, the average is probably the best known.  It is also easy to compute and in favorable cases, it provides a useful characterization of “the typical value” of a sequence of numbers.  It is not the only such “typical value,” however, nor is it always the most useful one: two other...

Read more »

Reading & plotting phylogenies

August 20, 2011
By

This is a basic procedure, but could come handy. I have been reading and doing a basic manipulations with phylogenetic trees a lot lately, so there is a chunk of code for this. > library (ape)   # ape is a … Continue reading →

Read more »

Statistical construction error

August 20, 2011
By
Statistical construction error

Yes, the title is meant to have two readings. The effect The Numbers Guy, among other examples, talks about the UK Office for National Statistics needing to revise its estimate for the construction sector output because of an error. Original: 2.3% growth Corrected: 0.5% growth Here is the Telegraph article cited by The Numbers Guy. … Continue reading...

Read more »

useR!2011

August 19, 2011
By
useR!2011

useR!2011 ended yesterday. First of all, much thanks to the organizers who managed to run a conference with 400+ participants, from 41 countries smoothly. Thumbs up! It was great to meet some people from the R blog-O-sphere in person, like … Continue reading →

Read more »

useR!2011

August 19, 2011
By
useR!2011

useR!2011 ended yesterday. First of all, much thanks to the organizers who managed to run a conference with 400+ participants, from 41 countries smoothly. Thumbs up! It was great to meat some people from the R blog-O-sphere in person, like Tal "R-blogg...

Read more »

Relation Between Fires and Distanse to the Nearest Highway

August 19, 2011
By
Relation Between Fires and Distanse to the Nearest Highway

Instead of introduction Just for fun I decided to investigate relationship between fires intensity in Leningrad region (and St. Petersburg as well) and distance to the nearest road in order to gain the evidence of the major influence of the anthropogen...

Read more »

display time series data in R

August 19, 2011
By
display time series data in R

Thanks to the Revolutions blog, several things learned here:1. R code for heat calendar2. generate SVG from R3. pretty-R toolOK. Let's explain it by plotting the fund WASCX (IVY ASSET STRATEGY FUND CLASS C) from 2009-03-14:# source code of calendarHeat...

Read more »

Friday quote: the handmaiden and the whore

August 19, 2011
By
Friday quote: the handmaiden and the whore

Because it is Friday and because we collect quotes: If mathematics is the handmaiden of science, statistics is the whore: all that scientists are looking for is a quick fix without the encumbrance of a meaningful relationship. Statisticians are second-class mathematicians, third-rate scientists and fourth-rate thinkers. They are the hyenas, jackals and vultures of the scientific ecology: picking...

Read more »

Friday quote: the handmaiden and the whore

August 19, 2011
By
Friday quote: the handmaiden and the whore

Because it is Friday and because we collect quotes: If mathematics is the handmaiden of science, statistics is the whore: all that scientists are looking for is a quick fix without the encumbrance of a meaningful relationship. Statisticians are second-class mathematicians, third-rate scientists and fourth-rate thinkers. They are the hyenas, jackals and vultures of the scientific...

Read more »

useR! 2011 roundup

August 19, 2011
By
useR! 2011 roundup

As I stand here at Heathrow waiting for my flight back to the States, I thought I'd dash off a few quick reflections of the userR! 2011 conference at University Warwick. It was an outstanding event. There's something about a conference of just a few hundred attendees (there were about 450) that creates a sense of camaraderie and common...

Read more »

Development of R (useR! 2011)

August 19, 2011
By
Development of R (useR! 2011)

Michael Rutter – R for Ubuntu Ubuntu 10.10 uses 2.10.1. Backports are newer versions of software for old releases. R backports are available CRAN (link). Lauchpad is a website for users to develop and maintain software (Canonical). One of Launchpad’s services is the personal package archive (PPA). This allows users to upload .deb source files, allowing

Read more »

R Function Binding Vectors and Matrices of Variable Length, bug fixed

August 19, 2011
By

Now this is something very geeky, but useful. I had to bind two matrices or vectors together to become a bigger matrix. However, they need not have the same number of rows or even the same row names. The standard cbind() functions require the vectors or matrices to be compatible. The matching is “stupid”, in

Read more »

useR2011 highlights

August 18, 2011
By
useR2011 highlights

useR has been exhilarating and exhausting. Now it’s finished, I wanted to share my highlights. 10. My inner twelve year old schoolgirl swooning and fainting with excitement every time I chatted with a member of R-core. 9. Patrick Burns declaring that his company consists of himself and his two cats. And that one of the

Read more »

Do older SOers use fewer words?

August 18, 2011
By
Do older SOers use fewer words?

On StackOverflow, to posters with more experience ask their questions in fewer words? No. There's no visible difference: Chars of non-code: Chars of code: The data comes from the super-handy StackOverflow API, which was retrieved using wget and then parsed using rjson and XML. First read in and parse the JSON: so.R 1...

Read more »

Halstead’s metrics and flat-Earthers are still with us

August 18, 2011
By
Halstead’s metrics and flat-Earthers are still with us

I recently discovered a fascinating series of technical reports from the 1970s in the Purdue University e-Pubs archive that shine a surprising light on what are now known as the Halstead metrics. The first surprises came from Halstead’s A Software Physics Analysis of Akiyama’s Debugging Data; surprising in the size of the data set used

Read more »

I, Rbot: Tweeting from R

August 18, 2011
By
I, Rbot: Tweeting from R

Over the past few weeks I’ve been running batches of JAGS simulations from R. Although these models typically converged within an hour or so, more complex models can take days, or even weeks to converge. Because we, as humans, are … Continue reading →

Read more »

HPC news from the useR2011 conference

August 18, 2011
By

It was an exciting useR2011 conference at the University of Warwick, Coventry, UK. Thanks a lot to the local organizing and program committee for having this great conference. I enjoyed the variety of talks, the poster session and the conference dinner and everything within walking distance. In view of HPC for R I learned: The

Read more »

Get Used To It

August 18, 2011
By
Get Used To It

This is a brain teaser. You've been warned.The S&P 500 is in a bear market (defined as the 50-day MA being below the 200-day MA) 30.8% of the time. Also, the S&P 500 has experienced single-day 4% declines 0.242% of the time. Of the times we exp...

Read more »

Simon Urbanek – R Graphics: supercharged

August 18, 2011
By
Simon Urbanek – R Graphics: supercharged

New features: rasterImage() (R2.11) bitmap raster drawing; have maps as data backdrops. Polygons with holes: polypath() -(R2.12) At present there is no way to tell when to actually show the plot. For example: plot(x); lines(x). Should we display the plot after plot or after lines Solution dev.hold() and dev.flush() Better performance and useful for animations –

Read more »

Installing RStudio Server on Scientific Linux 6: My bash notebook

August 18, 2011
By
Installing RStudio Server on Scientific Linux 6: My bash notebook

Granted, not a brilliant sysadmin mind at work here, but this might help someone someday.Scientific Linux (SL) is built from Red Hat Enterprise LinuxSee installation instructions here:http://rstudio.org/download/server$ sudo rpm -Uvhhttp://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-5.noarch.rpm password for leipzig: Retrievinghttp://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-5.noarch.rpmwarning: /var/tmp/rpm-tmp.S2RQAH: Header V3 RSA/SHA256 Signature, key ID0608b895: NOKEYPreparing... ...

Read more »

Kaleidoscope IIIb (useR! 2011)

August 18, 2011
By
Kaleidoscope IIIb (useR! 2011)

O. Mersmann - The microbenchmark package Slides and code (link). SURGEON GENERAL’s WARNING: Microbenchmarks can lead to a distorted view of reality and massive loss of productivity For a higher-order benchmarking package check out the rbenchmark package on R (suggestion from the speaker). Why do we need micro-benchmarking? A simple example showed that it is currently very

Read more »

Big data (useR! 2011)

August 18, 2011
By
Big data (useR! 2011)

Unfortunatley, I missed the first and last talks. My notes from a session on Thursday morning J. Demmler – Challenges of working with a large database of routinely collected health data The SAIL data bank holds over 1.9 billion (anonymous) entries. To use the data for research, they need to ensure that proper data security is

Read more »

We keep breaking records ? so what ?… Get statistical perspective….

August 17, 2011
By
We keep breaking records ? so what ?… Get statistical perspective….

This summer, we have been told that some financial series broke some records (here, in French) For instance, the French CAC40 had negative return for 11 consecutive days (which has never been seen, so far). > library(tseries)> x<-get....

Read more »

The stupidest R code ever

August 17, 2011
By
The stupidest R code ever

Let’s start this blog off right, with the stupidest R mistake I’ve ever made (I think). In the R package that I write, R/qtl, one of the main file formats is a comma-delimited file, where the blank cells in the second row are important, as they distinguish the initial phenotype columns from the genetic marker

Read more »