Suppose you are reading a paper that uses Likert scale responses. The paper reports the mean, standard deviation, and number of responses. If we are -- for some reason -- suspicious of a paper, we might ask, "Are these summary statistics possible for this number of responses, for this Likert ... [Read more...]

Suppose you are reading a paper that uses Likert scale responses. The paper reports the mean, standard deviation, and number of responses. If we are -- for some reason -- suspicious of a paper, we might ask, "Are these summary statistics possible for t... [Read more...]

[social4i size=”large” align=”float-right”] deadline: 15th of April   submission form and [Read more...]

I will be an invited speaker at GTC 2016, a large conference on GPU computation. The main topic will be usage of GPU in conjunction with R, and I will also speak on my Software Alchemy method, especially in relation to GPU computing.. GTC asked me to notify my “network” about ... [Read more...]

[Read more...]

by Bob Horton, Senior Data Scientist, Microsoft This is a follow-up to my earlier post on learning curves. A learning curve is a plot of predictive error for training and validation sets over a range of training set sizes. Here we’re using simulated data to explore some fundamental relationships ... [Read more...]

[Read more...]

[Read more...]

[caption id="attachment_1837" align="aligncenter" width="640"] As things change over time so should our statistical models. The image is CC by Prad Prathivi[/caption] Since I'm frequently working with large datasets and survival data I often find that the proportional hazards assumption for the Cox regressions doesn't hold. In my ... [Read more...]

Adaboosting is proven to be one of the most effective class prediction algorithms. It mainly consists of an ensemble simpler models (known as “weak learners”) that, although not very effective individually, are very performant combined. The process by which these … Continue reading → [Read more...]

A very nice puzzle on The Riddler last week that kept me busy on train and plane rides, runs and even in between over the weekend. The core of the puzzle is about finding the optimal procedure to select k guesses about the value of a uniformly random integer x ... [Read more...]

There's no denying that for a language as popular as R, it has more than its fair share of quirks. If you've ever wondered why, for example, R has a non-standard assignment operator, or that periods are allowed in symbols (and don't signify method calls), or that character data imports ... [Read more...]

Today I am happy to announce the R Election Analysis Contest. The goal of the contest is to encourage and promote high quality reproducible research in R that focuses on elections. The winner will be featured on my blog and receive a free copy of my course Mapmaking in R ... [Read more...]

Organize your data manipulation tasks in a standard way, write clean and efficient code, and build reproducible data management processes, using the most modern R tools: tidyr, dplyr and lubridate. The post "Efficient Data Manipulation with R" Course | April 11-12 Milan appeared first on MilanoR. [Read more...]

This note is a shameless plug demo of the ggnetwork package, which provides several geoms to plot network objects with ggplot2, and which just got published on CRAN. See the package vignette for a more detailed guide to its functionalities. Our example data is the most recent version of the ... [Read more...]

The March 28 issue of Bloomberg Businessweek has a rather good summary of the problems of p-values, even recommending the use of confidence intervals and — wonder of wonders — “[looking] at the evidence as a whole.” What, statistics can’t make our decisions for us?  :-) It does make some vague and ... [Read more...]

Almost exactly 20 years ago I wrote a paper with Yanan Fan on how sample quantiles are computed in statistical software. It was cited 43 times in the first 10 years, and 457 times in the next 10 years, making it my third paper to receive 500+ citations... [Read more...]

I previously did a short review on Microsoft’s image recognition and face detection API. A couple of weeks ago Google announced their vision API providing some similar features. Even though there is no R package or code to dive into this API and their API documentation is rather sparse, ... [Read more...]

The Delaware River experienced some high flow in late February 2016, providing an opportunity for an interesting animated graph of river response. This plot was developed using data from the USGS NWIS system for gauges on the Delaware River, retrieved with the excellent dataRetrieval library for R from USGS and the ... [Read more...]

The Delaware River experienced some high flow in late February 2016, providing an opportunity for an interesting animated graph of river response.This plot was developed using data from the USGS NWIS system for gauges on the Delaware River, retrieved w... [Read more...]