Getting backgammon data out of XG-Gammon Backgammon is a game that combines chance and skill, and everyone who comes across it asks “how much is luck, and how much skill?”. The answer of course is “it depends”. Consider that for two exactly equally skilled players, the result appears to be 100% ... [Read more...]

Control charts I wanted to help explore the implications of changing sample size for a quality control process aimed at determining the defect rate in multiple sites. Defect in this particular case is binary ie the products are either good or not. Much of the advice on this in the ... [Read more...]

Just a short note that in my day job we’ve released the New Zealand Tourism Dashboard, launched by the Associate Minister for Tourism earlier today. It’s built with R and Shiny and I won’t say more about it than that to avoid mixing up my work and ... [Read more...]

General Equilibrium economic models To expand my economics toolkit I’ve been trying to get my head around Computable General Equilibrium (CGE) and Dynamic Stochastic General Equilibrium (DSGE) models. Both classes of model are used in theoretical and policy settings to understand the impact of changes to an economic system ... [Read more...]

In a post a few months ago I built a new ggplot2 statistical transformation (“stat”) to provide X13-SEATS-ARIMA seasonal adjustment on the fly. With certain exploratory workflows this can be useful, saving on a step of seasonally adjusting many different slices and dices of a multi-dimensional time series during ... [Read more...]

Eric Persson released yesterday a new WHO R package which allows easy access to the World Health Organization’s data API. He’s also done a nice vignette introducing its use. I had a play and found it was easy access to some interesting data. Some time down the track ... [Read more...]

Forecast Combination I’ve referred several times to this blog post by Rob Hyndman in which he shows that a simple averaging of the ets() and auto.arima() functions in his {forecast} R package not only out performs ets() and auto.arima() individually (in the long run, not every time), ... [Read more...]

Individual-level estimates from survey data I was motivated by web apps like the British Office of National Statistics’ How well do you know your area? and How well does your job pay? to see if I could turn the New Zealand Income Survey into an individual-oriented estimate of income given ... [Read more...]

Twitter bots One of my holiday projects was to get my head around how Twitter bots work, and the best (only?) way of course is to make one. Apparently about one in seven Twitter accounts is a computer program rather than a human user (although of course there are humans ... [Read more...]

Commuting between districts and cities in New Zealand At this year’s New Zealand Statisticians Association conference I gave a talk on Modelled Territorial Authority Gross Domestic Product. One thing I’d talked about was the impact on the estimates of people residing in one Territorial Authority (district or city) ... [Read more...]

The M3 forecasting competition The M3 forecasting competition in 2000, organized by Spyros Makridakis and Michele Hibon, tested a variety of methods against 3,003 time series, with forecasts compared to held out test sets. The data are conveniently available for R users in the Mcomp package and Rob Hyndman has published example ... [Read more...]

Exports and economic growth I was looking to show a more substantive piece of analysis using the World Development Indicators data, and at the same time show how to get started on fitting a mixed effects model with grouped time series data. The relationship between exports’ importance in an economy ... [Read more...]

World Development Indicators After my last post on deaths from assault, I got a few comments both on and off-line asking for exploration of developing country issues, in contrast to the OECD focus of the previous post. I’d always intended at some point to do something with the World ... [Read more...]

Deaths from assault A friend was looking at comparative data on violent deaths in a range of countries and I couldn’t resist the chance to turn out some graphics. The data come from the OECD, who compiles them from contributions by member and associated country governments - effectively, wealthier ... [Read more...]

Simulating ARIMA models Generating an arbitrary Auto-Regressive Integrated Moving Average (ARIMA) model is easy in R with the arima.sim() function that is part of the built-in {stats} package. In fact I've done it extensively in previous blog posts for various illustrative purposes. But one cost of doing this for ... [Read more...]

What do auto-correlated residuals do to your linear model? For training purposes I wanted to illustrate the dangers of ignoring time series characteristics of the random part of a classical linear regression, and I came up with this animation to do it: I like this, because it shows how easy ... [Read more...]

A big project at my work that I was involved with over the past year was the production of modelled estimates of gross domestic product at the District and City level. There are official statistics at the Regional Council level (16 of them in New Zealand), so the aim of this ... [Read more...]

Silver trade I’ve been reading Ronald Findlay and Kevin H. O’Rourke’s 2009 book Power and Plenty: Trade, War, and the World Economy in the Second Millennium and finding it an excellent piece of big picture economic history. I felt the urge to re-express one of their figures (Figure 4.7 ... [Read more...]

Calling seasonal adjustment software from R I recently explored for the first time (having languished on the “check this out later” list) Christoph Sax’s excellent {seasonal} R package. It makes it super easy for R users to engage with X-13ARIMA-SEATS, the latest industry standard software for time series ... [Read more...]

The jobs Within the New Zealand Ministry of Business, Innovation and Employment, the Sector Trends team has recently secured resourcing for additional analysts on a range of statistical programmes. That’s the team that I usually manage, although for the next few months I’m doing a stint on a ... [Read more...]