In Accessing and Visualising Sentencing Data for Local Courts I posted a couple of quick ways in to playing with Ministry of Justice sentencing data for the period July 2010-June 2011 at the local court level. At the end of the post, I wondered about how to wrangle the data in ... [Read more...]

A recent provisional data release from the Ministry of Justice contains sentencing data from English(?) courts, at the offence level, for the period July 2010-June 2011: “Published for the first time every sentence handed down at each court in the country between July 2010 and June 2011, along with the age and ethnicity ... [Read more...]

[This article was first published on Freakonometrics - Tag - R-english, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here) Want [Read more...]

I’ve been working through Gelman et al.’s otherwise excellent Bayesian Data Analysis and it’s going reasonably well. My statistics is a little bit rusty so it’s taken time to work through all of the exercises and really understand what’s going on. But I say “otherwise ... [Read more...]

I will be in Lyon next Monday to give a talk on "Modeling heat-waves: return period for non-stationary extremes" in a workshop entitled "Changement climatique et gestion des risques". An interesting reference might be some pages from Le Monde (201... [Read more...]

Kay Cichini recently wrote a word-cloud R function called GScholarScraper on his blog which when given a search string will scrape the associated search results returned by Google Scholar, across pages, and then produce a word-cloud visualisation. This was of interest to me because around the same time I posted ... [Read more...]

I wanted to play around with the Facebook Graph API  using the Graph API Explorer page as a coding exercise. This facility allows one to use the API with a temporary authorisation token. Now, I don’t know how to make an R package for the proper API where you ... [Read more...]

Earlier today, I saw a post vis the aggregating R-Bloggers service a post on Using Text Mining to Find Out What @RDataMining Tweets are About. The post provides a walktrhough of how to grab tweets into an R session using the twitteR library, and then do some text mining on ... [Read more...]

This is a followup to a post I uploaded earlier today about web scraping data off Google Scholar. In that post I was frustrated because I’m not smart enough to use xpathSApply to get the kind of results I wanted. However fast-forward to the evening whilst having dinner with ... [Read more...]

I wanted to scrape the information returned by a Google Scholar web search into an R data frame as a quick XPath exercise. The following will successfully extract  the ‘title’, ‘url’ , ‘publication’ and ‘description’.  If any of these fields are not available, as in the case of a citation, the ... [Read more...]

Reading our local weekly press this evening (the Isle of Wight County Press), I noticed a page 5 headline declaring “Alarm over death rates at St Mary’s”, St Mary’s being the local general hospital. It seems a Department of Health report on hospital mortality rates came out earlier this ... [Read more...]

Consider a (simple) Poisson regression . Given a sample where , the goal is to derive a 95% confidence interval for given , where is the prediction. Hence, we want to derive a confidence interval for the prediction, not the potential observation... [Read more...]

Earlier this year I came across a rather interesting page about optimisation in R from rwiki. The goal was to find the most efficient code to produce strings which follow the pattern below given a single integer input n: From this we can see that the general pattern for n ... [Read more...]

The Gaussian vector is extremely interesting since it remains Gaussian when conditioning. More precisely, if is a Gaussian random vector, then the conditional distribution of is also Gaussian. Further, it is possible to derive explicitly the cova... [Read more...]

If part of the role of data journalism is to make transparent the justification behind claims that are, or aren’t, backed up by data, there’s good reason to suppose that the journalists should be able to back up their own data-based claims with evidence about how they made ... [Read more...]

Picking up on Paul Bradshaw’s post A quick exercise for aspiring data journalists which hints at how you can use Google Spreadsheets to grab – and explore – a mortality dataset highlighted by Ben Goldacre in DIY statistical analysis: experience the thrill of touching real data, I thought I’d describe ... [Read more...]

I've knocked together a quick function for generating efficient Monte Carlo samples. It takes a bit of the legwork out of running Monte Carlo simulations. [Read more...]

Recently, I have been reading odd things about strategies to win at the lottery. E.g. or I wrote something a long time ago, but maybe it would be better to write another post. First, it is easy to get data on the French lotteries, including dra... [Read more...]

Yesterday, I wrote a post (in French) about short-selling in financial market since some journalists claimed that it was well-known that short -selling does increase volatility on financial market. Not only in French speaking journals actually, sin... [Read more...]

Teacher: “How variable is your estimate of the mean?” Student: “Uhhh, it’s not. I took a sample and calculated the sample mean. I only have one number.” Teacher: “Yes, but what is the standard deviation of sample means?” Student: “What do you mean means, I only have the one ... [Read more...]