I have been playing with R to parse html. After reading about visualising “fantasy football” search traffic with RGoogleTrends at The Log Cabin blog I decided to write a few functions to do similar things with Wikipedia search statistics. This … Continue reading → [Read more...]

Programmers should definitely know how to use R. I don’t mean they should switch from their current language to R, but they should think of R as a handy tool during development.Again and again I find myself working with Java code like the following. td.linenos { background-color: #f0... [Read more...]

I got a paper (unavailable online) to referee about testing for the order (i.e. the number of components) of a normal mixture. Although this is an easily spelled problem, namely estimate k in I came to the conclusion that it is a kind of ill-posed problem. Without a clear ... [Read more...]

My love for template method pattern hit the wall. The wall of Java constructors that is. Do not mix them together ever or at least without double checking. Again, I am all for template method pattern when it operates on fully initialized objects. But, ... [Read more...]

Revolution Analytics is hosting several hands-on R training classes over the next few months, with in-person instruction from two leading package authors and experts from the R community. Diethelm Würtz from ETH Zurich will give a two-day master class on Portfolio Selection and Optimization in Practice. Prof Würtz ... [Read more...]

Several years ago Gerd Gigerenzer wrote: “Statistical rituals largely eliminate statistical thinking in the social sciences. Rituals are indispensable for identification with social groups, but they should be the subject rather than the procedure of science. Statistical rituals largely eliminate … Continue reading → [Read more...]

Ever have a regression model where the coefficients don't make sense? I've been trying to predict electricity and gas consumption from daily activity schedules but a simple linear regression kept saying that demands should go down the more an activity is performed. Fortunately I found the nnls package and show ... [Read more...]

While my time at the 2011 Joint Statistical Meetings was short--I unfortunately missed some presentations I would have like to have attended--it was a great experience. The collection of academics and professionals is very different from the other con... [Read more...]

In recent years many R packages have been developed to enable image analysis in R. As an alternative the combination of R with a powerful image analysis software like ImageJ offers many advanced image analysis interfaces and algorithms not yet available in R. Bio7 integrates both applications in a Rich ... [Read more...]

Here are the 14 slides I used during my talk at the Joint Statistical Meetings 2011: shotwell-jsm-2011.pdf. I'm trying hard to minimize the text in my presentation slides. But, this usually requires that I practice more. Hence, you will know which talks I have practiced thoroughly by the amount of text ... [Read more...]

[Read more...]

Together with Revolution Analytics, I will be offering two more one-day classes on the Rcpp package for seamless integration of R and C++. The format will follow the workshop Romain and I gave during the tutorial day preceding this year's R/Financ... [Read more...]

[Read more...]

A reader asked a question about data from   environment canada.  He wanted to know if that data could somehow be integrated into the RGhcnV3 package.  That turned out to be a bit more challenging that I expected.  In short order I’d found a couple other people who had done ... [Read more...]

At the JSM 2011 conference in Miami earlier this week, we conducted an informal poll of attendees on their attitudes to respect to Big Data, statistical software, and data science. JSM is the largest gathering of statisticians in North America, and attendees were invited to complete a survey after logging into ... [Read more...]

[Read more...]

Of course since we all know Jon Skeet does have various powers, I will move onto unanswered questions, whether a users reputation makes them receive more upvotes for answers. I’ve seen this theory mentioned in multiple places (see any of the comments to Jon Skeet’s answer that are ... [Read more...]

I'm in the process of putting together some slides on how to apply Quantile-Quantile plots to performance data. Q-Q plots are a handy tool for visually inspecting how well your data matches a known probability distribution (prob dsn). If the match is g... [Read more...]

We have an internal image that floated around work several years ago that details network utilization of TCP over a wide variety of configurations. It is a heatmap created in matlab that is just sweet, sweet eye candy. We actually hung it on the outside of a cube for a ... [Read more...]

At JSM 2011 today, three Google employees (amongst the more than 20 Google delegates there) gave a little insight into how statistical analysis with R yields better results for companies using Google's various advertising products. Bill Heavlin from Google kicked off the session with a talk about conditional regression models, a statistical ... [Read more...]