I am so lucky to work with so many generous, knowledgeable, and amazing people at Stack Overflow, including Ian Allen and Kirti Thorat. Both Ian and Kirti are part of biweekly sessions we have at Stack Overflow where several software developers join me in practicing R, data science, and modeling skills. This morning, the two of them went to...
Prediction competitions are now so widespread that it is often forgotten how controversial they were when first held, and how influential they have been over the years. To keep this exercise manageable, I will restrict attention to time series forecast...
Tal Galili's R-bloggers.com has been syndicating blog posts about R for quite a while — from memory I'd say about 8 years, but I couldn't find the exact date it started aggregating. Anyway, it contains a wealth of information about activity in the R ecosystem, but without any easy way to access that information other than the blog post...
Udemy is offering readers of R-bloggers access to its global online learning marketplace for $15 per course! This deal (offering over 50%-90% discount) is for hundreds of their courses – including many R-Programming, data science, machine learning etc. Click here to browse ALL (R and non-R) courses Advanced R courses: Regression modelling Comprehensive Linear Modeling with R (15 Hours of video) Linear regression in R...
Now that software ate the world, what’s for dessert? Those in the know know that it’s AI. It seems everyone …Continue reading →
R tip. Use wrapr::match_order() to align data. Suppose we have data in two data frames, and both of these data frames have common row-identifying columns called “idx“. library("wrapr") d1
In January, I was excited to make an announcement about a shift in my career: I have some exciting news: today I'm joining @DataCamp as their Chief Data Scientist 🎉📊📈 pic.twitter.com/wiN9J4qSjx— David Robinson (@drob) January 29, 2018 When I first discussed the role with the DataCamp CEO, I described my goal as to “Make DataCamp as good at doing data science...
Data science gives you tools to find opportunities A few months ago, I wrote a blog post about Amazon’s search for a location for a second headquarters. For those of you who don’t know about it, the technology giant – which currently has its headquarters in Seattle Washington – announced last September that it The post Mapping the...
A conundrum for Microsoft Power BI I’ve been familiarising myself with Microsoft Power BI, which features prominently in any current discussion on data analysis and dissemination tools for organisations. It’s a good tool with some nice features. I won’t try to do a full review here, but just ruminate on one aspect - setting it up for non-specialists...
I have added a new function to my {brotools} package, called describe(), which takes a data frame as an argument, and returns another data frame with descriptive statistics. It is very much inspired by the {skmir} package but also by assist::describe() (click on the packages to be redirected to the respective Github repos) but I wanted to write my...
In my last post I started to build two functions that took pairs of variables from a dataset and produced some nice useful ggplot plots from them. We started with the simplest case, plotting counts of how two variables cross-tabulate. Then we worked our way up to being able to automate the process of plotting lots of pairings of...
INTRODUCTION Highcharter is a R wrapper for Highcharts javascript libray and its modules. Highcharts is very mature and flexible javascript charting library and it has a great and powerful API. Before proceeding, please follow our short tutorial. Look at the examples given and try to understand the logic behind them. Then, try to solve the Related exercise sets:Data Visualization...
The Herfindahl-Hirschman Index (HHI) is a widely used measure of concentration in a variety of fields including, business, economics, political science, finance, and many others. Though simple to calculate (summed squared market shares of firms/actors in a single market/space), calculation of the HHI can get onerous, especially as the number of firms/actors increases and the … Continue reading A...
Recently, a paper was presented at our university that showed a significant effect for a variable of interest but had a relatively small number of observations. One colleague suggested that we should consider the significance of the results with care since the number of observations was fairly small. This ignited some discussion. Given that the significance test computed exact p-values,...
TL;DR: Use the Vectorize() function! If you’re reading this, you’ve either encountered this problem before, or you just got to this article out of curiousity (in which case you probably don’t know what problem I’m talking about). A few days ago I was given code by a client for a function that, given a path to a patient’s file, generates a...
Join RStudio Chief Data Scientist Hadley Wickham for his popular Building tidy tools workshop in San Francisco! If you’d missed the sold out course at rstudio::conf 2018 now is your chance. Register here: https://www.rstudio.com/workshops/extending-the-tidyverse/ You should take this class if you have some experience programming in R and you want to learn how to tackle larger scale problems. You’ll get...
Notes about creation of Harry Potter Books Survey. It is not over, I need your help. Prologue Right now I am in the final stage of developing two packages devoted to results of abstract competitions (still not perfectly ready, so use with caution): comperes - infrastructure package for dealing with different formats...
This post is to announce a second occurrence of the exciting “masterclass in Bayesian Statistics” that we organised in 2016, near Marseilles. It will take place on 22-26 October 2018 once more at CIRM (Centre International de Recherches Mathématiques, Luminy, Marseilles, France). The targeted audience includes all scientists interested in learning how Bayesian inference may
In a couple of recent posts (Textualisation With Tracery and Database Reporting 2.0 and More Tinkering With PyTracery) I’ve started exploring various ways of using the pytracery port of the tracery story generation tool to generate variety of texts from Python pandas data frames. For my F1DataJunkie tinkerings I’ve been using R + SQL as
A new release of the tint package just arrived on CRAN. Its name expands from tint is not tufte as the package offers a fresher take on the Tufte-style for html and pdf presentations. This version adds support for the tufte-book latex style. The pack...
Many R package authors (including myself) lump a collection of small, useful functions into some type of utils.R file and usually do not export the functions since they are (generally) designed to work on package internals rather than expose their functionality via the exported package API. Just like Batman’s utility belt, which can be customized... Continue reading →
This exercise set will continue to present the STAN platform, but with another useful tool: the bayesplot package. This package is very useful to construct diagnostics that can be used to have insights on the convergence of the MCMC sampling since the convergence of the generated chains is the main issue in most STAN models. Related exercise sets:Spatial Data...
We recently had an awesome opportunity to work with a great client that asked Business Science to build an open source anomaly detection algorithm that suited their needs. The business goal was to accurately detect anomalies for various marketing data consisting of website actions and marketing feedback spanning thousands of time series across multiple customers and web sources. Enter...
Some time ago we discussed how you can access data that are stored in Amazon Redshift and PostgreSQL with Python and R. Let’s say you did find an easy way to store a pile of data in your BigQuery data warehouse and keep them in sync. Now you want to start messing with it using statistical techniques, maybe build...
A bit more tinkering with F1 data from the ergast db, this time trying to generating trivia / facts around races. The facts are identified using SQL queries: Some of the queries also embed query fragments, which I intend to develop further… I'm using knitr to generate Github flavoured markdown (gfm) from my Rmd docs
In this exercise, we will continue to build our model from our previous exercise here, specifically to revise the errors that may be generated from the model, including rounding and truncating errors. Answers to these exercises are available here. If you obtained a different (correct) answer than those listed on the solutions page, please feel Related exercise sets:3D plotting...
Someone recently said to me that the lower the p-value, the higher the likelihood ratio under the alternative vs the null. The arXiv paper by Michael Lew makes analogous points (thanks to Titus von der Malsb...
Let’s consider piping in R both using the magrittr package and using the wrapr package. magrittr pipelines The magittr pipe glyph “%__%” is the most popular piping symbol in R. magrittr documentation describes %__% as follow. Basic piping: x %__% f is equivalent to f(x) x %__% f(y) is equivalent to f(x, y) x %__% … Continue reading magrittr...
I’m back on the R thang with F1 data from ergast.com, and started having a look at how drivers and teams compare at a circuit. One metric I came across for comparing teams over a season is the supertime, typically calculated for each manufacturer as the average of their fastest single lap recorded by the team
