Back in 2010, I wrote a web application called PMRetract to monitor retraction notices in the PubMed database. It was written primarily as a way for me to explore some technologies: the Ruby web framework Sinatra, MongoDB (hosted at MongoHQ, now Compose) and Heroku, where the app was hosted. I automated ... [Read more...]

Introduction:GithubThe Western Front in World War One is considered a a war of outdated tactics combined with brutal efficiency of technology. And while the war saw the introduction of several innovations (eg. tanks and airplanes) the general routine o... [Read more...]

Yesterday, I did upload a post where I tried to show that “standard” regression models where not performing bad. At least if you include splines (multivariate splines) to take into accound joint effects, and nonlinearities. So far, I do not discuss the possible high number of features (but with boostrap ... [Read more...]

[Read more...]

I saw a link to a list with all rocket launches into space the other day. This post contains some plots concerning day of launch made from that.DataData is a fixed format file with eleven columns. Reading fixed format is not very difficult, however, it... [Read more...]

Motivation R has excellent for dates and times via the built-in Date and POSIXt classes. Their usage, however, is not always as straightforward as one would want. Certain conversions are more cumbersome than we would like: while as.Date("2015-03-22"), would it not be nice if as.Date("20150322") (a ... [Read more...]

Motivation Using R within a latex document can be a component of reproducible research, offering (a) some assurance against typographical errors in transcribing results to the latex file and (b) the ability for others to reproduce the results. For example, one might like to explain how close the computed integral ... [Read more...]

As some of my regular readers may know, I'm in the middle of writing a book on introductory data analysis with R. I'm at the point in the writing of the book now where I have to make some hard… Continue reading → [Read more...]

I just published a new interactive visualization in my series of basic statistical concepts and techniques. This time I am trying to show how p-values are distributed. Check it out here: rpsychologist.com/d3/pdist/ [Read more...]

The tree-based Cubist model can be easily used to develop an ensemble classifier with a scheme called “committees”. The concept of “committees” is similar to the one of “boosting” by developing a series of trees sequentially with adjusted weights. However, the final prediction is the simple average of predictions from ... [Read more...]

[Read more...]

The On Broadway project collected more than 600,000 photographs taken near Broadway in New York City during a six-month period in 2014. If you're in New York, you can explore the images in an interactive installation at the New York Public Library though the end of this year. You can also explore ... [Read more...]

Features form the basis for much of our preference modeling. When asked to explain one's preferences, features are typically accepted as appropriate reasons: this job paid more, that candidate supports tax reform, or it was closer to home. We believe t... [Read more...]

[Read more...]

Another year, another web scraping course. Taught through SSRMC at the University of Cambridge. Below are slides from all three sessions.In the course I tried to achieve the following:- Show how to connect R to resources online- Use loops and functions... [Read more...]

[Read more...]

Every other year Rexer Analytics surveys Data Analysts, Predictive Modelers, Data Scientists, Data Miners, and all other types of analytic professionals, students, and academics regarding the software they use.  I then update the main results in The Popularity of Data Analysis … Continue reading → [Read more...]

A while back I wrote a post on using the R tool tapply, which is my most popular post. Tapply allows you to repeat a function over a vector split into groups, e.g.… Continue reading → [Read more...]

Geomorph users,Our function plotTangentSpace() performs a Principal Components Analysis (PCA) of shape variation and plots two dimensions of tangent space for a set of Procrustes-aligned specimens and also returns the shape cha... [Read more...]

Cohen is power. Inferential statistics primarily invoke the following four key concepts: sample size, significance criterion, effect size, and statistical power. Cohen elegantly developed the maths, benchmarks, and key semantics associated with statistical power.   Statistical power is the long-term probability of rejecting the null hypothesis (typically assumed to be no ... [Read more...]