The problem of outliers – data points that are substantially inconsistent with the majority of the other points in a dataset – arises frequently in the analysis of numerical data.  The practical importance of outliers lies in the fact that even a few of these points can badly distort the results of ... [Read more...]

I've used both R and Stata for a long time, but these days I use Stata much more frequently than R. While R is useful for some kinds of graphics (especially three-dimensional graphics) and some statistical procedures (for example, finite mixture models... [Read more...]

[Read more...]

The german DIN33430 defines quality standards that must be met in “job-related proficiency assessments”, the qualifications of the responsible parties involved, as well as the creation, execution and evaluation of such assessments. Licensed persons are published on a website (german). … Weiterlesen → [Read more...]

Hadley Wickham has just released an update to the ggplot2 graphics package for R. Version 0.9.0 significantly speeds up the process of rendering graphics, and the documentation is much improved (including the addition of many new examples). This update also adds a bunch of new features, which are documented in this 40... [Read more...]

[Read more...]

MENTALLY MULTIPLY NUMBERS BY THEMSELVES Assume you know your multiplication tables up to 10x10. Here's how to compute the squares of numbers from 11 to 100. [Read more...]

I recently thought about ways to visualize medications and their co-occurences in a group of children. As long as you want to visualize up to  4 different medications you can simply use Venn diagrams. There is a very nice R-package to generate these kind of graphics for you (for a  description ... [Read more...]

I've only just started looking at the homicide data made available by the Philadelphia Inquirer in my free time (which is hard to come by lately). I've been thinking about what sorts of statistics I could do, or what kinds of additional data sets I cou... [Read more...]

Here are the R codes of the second R lab organised by Serena Arima in supplement of my lectures (now completed!). This morning I covered ABC model choice and the following example is the benchmark used in the course (and in the paper) about the impact of summary statistics. (Warning! ... [Read more...]

Much of the data that the analyst uses exhibits extraordinary range. For example: incomes, company sizes, popularity of books and any “winner takes all process”; (see: Living in A Lognormal World). Tukey recommended the logarithm as an important “stabilizing transform” (a transform that brings data into a more usable form ... [Read more...]

This is the first post of a series that describes how to download and parse specific data sets into R. These kinds of scripts can be functionalized further, but I doubt that these will ever find their way into a formal package. They are intended to be helpful to those ... [Read more...]

[Read more...]

This is yet another experiment to see how good is the approximation of binomial probability when we use Poisson and normal distributions for scenarios with large $n$, and $p$ close to zero or one. Consider a problem where the random variable $X$ follows a binomial distribution with a known probability ... [Read more...]

Insurance pricing is backwards and primitive, harking back to an era before computers. One standard (and good) textbook on the topic is Non-Life Insurance Pricing with Generalized Linear Models by Esbjorn Ohlsson and Born Johansson. We have been doing some work in this area recently. Needing a robust internal training ... [Read more...]

[Read more...]

In the previous post I shared an example how to get high frequency data from IB broker (well, it is retail version of HFD – it has only best bid/ask and the trades). Now, once you saved some data – what should you do next? Next logical step would be data ... [Read more...]

Ben Goldacre, the physician and biostatistician behind the always-excellent Bad Science column in the Guardian, gave a barnburner of a talk at Strata 2012 yesterday, "The Information Architecture of Medicine is Broken". For anyone not aware of the problems caused by publication bias in clinical trials (for example, ineffective drugs with ... [Read more...]

May 8, 2012 - 18:00 - 21:00 Fiori Oscuri Bistrot & Bar Via Fiori Oscuri, 3 - Milano (Zona Brera) Continue reading → [Read more...]

[Read more...]