In this post we are talking about one of the most unintuitive results of statistics: the so called false positive paradox which is an example of the so called base rate fallacy. It describes a situation where a positive test result of a very sensitive medical test shows that you ... [Read more...]

One of the problems of navigating an autonomous car through a city is to extract robust signals in the face of all the noise that is present in the different sensors. Just taking something like an arithmetic mean of all the data points could possibly end in a catastrophe: if ... [Read more...]

A few weeks ago we published a post about using the power of the evolutionary method for optimization (see Evolution works!). In this post we will go a step further, so read on… A problem researchers often face is that they have an amount of data and need to find ... [Read more...]

Asset returns have certain statistical properties, also called stylized facts. Important ones are: Absence of autocorrelation: basically the direction of the return of one day doesn’t tell you anything useful about the direction of the next day. Fat tails: returns are not normal, i.e. there are many more ... [Read more...]

As promised in the post Learning Data Science: Modelling Basics we will now go a step further and try to predict income brackets with real world data and different modelling approaches. We will learn a thing or two along the way, e.g. about the so-called Accuracy-Interpretability Trade-Off, so read ... [Read more...]

In this post we will see that a little bit of simple R code can go a very long way! So let’s get started! One of the fascinating features of number theory (unlike many other branches of mathematics) is that many statements are easy to make but the brightest ... [Read more...]

Hamlet: Do you see yonder cloud that’s almost in shape of a camel? Polonius: By the mass, and ’tis like a camel, indeed. Hamlet: Methinks it is like a weasel. from Hamlet by William Shakespeare The best way to see how evolution works, is to watch it in action! ... [Read more...]

One of the classic examples in data science (called data mining at the time) is the beer and diapers example: when a big supermarket chain started analyzing their sales data they encountered not only trivial patterns, like toothbrushes and toothpaste being bought together, but also quite strange combinations like beer ... [Read more...]

Sorting values is one of the bread and butter tasks in computer science: this post uses it as a use case to learn what recursion is all about. It starts with some nerd humour… and ends with some more, so read on! Have a look at the following code and ... [Read more...]

One of the topics that is totally hyped at the moment is obviously Artificial Intelligence or AI for short. There are many self-proclaimed experts running around trying to sell you the stuff they have been doing all along under this new label. When you ask them what AI means you ... [Read more...]

Data Science is all about building good models, so let us start by building a very simple model: we want to predict monthly income from age (in a later post we will see that age is indeed a good predictor for income). For illustrative purposes we just make up some ... [Read more...]

We are living in the era of Big Data but the problem of course is that the bigger our data sets become the slower even simple search operations get. I will now show you a trick that is the next best thing to magic: building a search function that practically ... [Read more...]

…the richest man on earth would have a fortune of no more than $43,000! If you don’t believe me read this post! Have you ever thought about the distribution of wealth as a function of some quality? Especially rich people pride themselves on extraordinary abilities, so that they somehow “deserve” ... [Read more...]

Everything “neural” is (again) the latest craze in machine learning and artificial intelligence. Now what is the magic here? Let us dive directly into a (supposedly little silly) example: we have three protagonists in the fairy tail little red riding hood, the wolf, the grandmother and the woodcutter. They all ... [Read more...]

Noseman is having a headache and as an old-school hypochondriac he goes to see his doctor. His doctor is quite worried and makes an appointment with a radiologist for Noseman to get a CT scan. Because Noseman always wants to know how things work he asks the radiologist about the ... [Read more...]

[Read more...]

During this time of year there is obviously a lot of talk about the Bible. As most people know the New Testament comprises four different Gospels written by anonymous authors 40 to 70 years after Jesus’ supposed crucifiction. Unfortunately we have lost all of the originals but only retained copies of copies ... [Read more...]

I want to draw your attention to a very valuable (and short!) whitepaper from my colleague, Professor Andrew Ng, where he shares important insights on how to lead companies into the AI era. The five steps outlined in the paper are: Execute pilot projects to gain momentum Build an in-house ... [Read more...]

Have you ever thought about why the definition of a function in R is different from many other programming languages? The part that causes the biggest difficulties (especially for beginners of R) is that you state the name of the function at the beginning and use the assignment operator – as ... [Read more...]

Principal component analysis (PCA) is a dimension-reduction method that can be used to reduce a large set of (often correlated) variables into a smaller set of (uncorrelated) variables, called principal components, which still contain most of the information. PCA is a concept that is traditionally hard to grasp so instead ... [Read more...]