The Eye of the World is the first book of Robert Jordan's Wheel of Time books. As the last of these books will be published soon, I was wondering if natural language processing can be used to examine books like these. For this purpose I downloaded a co... [Read more...]

I was having a conversation with an acquaintance about courses that were particularly useful in our work. My forestry degree involved completing 50 compulsory + 10 elective† courses; if I had to choose courses that were influential and/or really useful they would be Operations Research, Economic Evaluation of Projects, Ecology, 3 Calculus and 2 ... [Read more...]

lme4 and its cousin arm are extremely useful for a huge variety of modeling applications (see Gelman and Hill’s book), but today we’re going to do something a little frivolous with them. Namely, we’re going to extend our Denver Debat... [Read more...]

Download complete issue Refereed articles may be downloaded individually using the links below. [Bibliography of refereed articles] Table of Contents Editorial 3   [Read more...]

I’ve recently been considering the graphical presentation of data. I get the feeling that we, ecologists/scientsits, could be better at data presentation. Graphs must be informative, but they don’t have to be ugly. I think that making visually appealing charts … Continue reading → [Read more...]

Recently whiling trying to build an R package for generalized estimating equation model selection (QICpack on github), I was getting an error related to latex creating the PDF package manuals. It seems like this is a relatively common problem on … Continue reading → [Read more...]

The October 2012 issue of Harvard Business Review prominently features the words “Getting Control of Big Data” on the cover, and the magazine includes these three related articles:“Big Data: The Management Revolution,” by Andrew McAfee and Erik Brynjolfsson, pages 61 – 68;“Data Scientist: The Sexiest Job of the 21st Century,” by Thomas ... [Read more...]

Today’s Gist takes the CNN transcript of the Denver Presidential Debate, converts paragraphs into a document-term matrix, and does the absolute most basic form of text analysis: a raw word count. There are actually quite a few steps in this proc... [Read more...]

Here is the mathematical puzzle of the weekend edition of Le Monde: Consider a sequence where the initial number is between 1 and 10³, and each term in the sequence is derived from the previous term as follows: if the last digit of the previous term is between 6 and 9, multiply it by 9; [...] [Read more...]

Together with Mateusz Zawisza I have just published a new book in Polish on building predictive models in GNU R. It can be bought at Oficyna Wydawnicza SGH. The book presents complete examples of basic data mining processes.Although the book is in Poli... [Read more...]

[Read more...]

The PMPATH particle tracking output, with a file format similar to the pathline output mode of MODPATH (see above), can be transformed easily into a GIF animation using R and ImageMagick (see below for a simple example).First of all, you... [Read more...]

The most recent edition of the Revolution Newsletter is out. The news section is below, and you can read the full December edition (with highlights from this blog and community events) online. You can subscribe to the Revolution Newsletter to get it monthly via email. Tell us what you're looking ... [Read more...]

[Read more...]

Introduction A few months ago, Drew Conway and I gave a webcast that tried to teach people about the basic principles behind linear and logistic regression. To illustrate logistic regression, we worked through a series of progressively more complex spam detection problems. The simplest data set we used was the ... [Read more...]

One of the first weird things to get used to in R is unlearning some of the things that you think you know. As often happens, this reminds me of a quote I once read about Zen, which went about like this (I’m paraphrasing), “When I knew nothing of ... [Read more...]

A couple days ago I noticed a fun piece of R code by Allan Roberts, which lets you create a digital snowflake by cutting out virtual triangles. Go give it a try. Roberts inspired me to create a whole night sky of snowflakes. I tried to make the snowfall look ... [Read more...]

I'm happy to announce that my course Computing for Data Analysis will return to Coursera on January 2nd, 2013. While I had previously announced that the course would be presented again right here, it made more sense to do it … Continue reading → [Read more...]

This post is tangential to R, although R has a fair share of the issues I mention here, which include research reproducibility, open source, paying for software, multiple languages, salt and pepper. There is an increasing interest in the reproducibility … Continue reading → [Read more...]

If you have your working environment set up in a Windows operating system, it can be a bit of a hassle to work with R sessions on remote Linux servers.I use WinSCP + Notepad++ to handle my projects and Putty + screen to handle the R sessions. It become... [Read more...]