I know I am probably late to this party but I recently found out about DBSCAN or "A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise"[^1]. In a nutshell, the algorithm visits successive data point and asks whether neighbouring points are density-reachable. In other words is it ... [Read more...]

Just disovered the Jekyll - github pages combination. Perfect for a very simple static blog site. I also found a pretty neat integration with RMarkdown which suits me perfectly, as I am mainly using R and will be posting bits and pieces of that. A summary of how it works... ... [Read more...]

Recently, I came across this great visualization of MLS Player salaries. I tried to do something similar with ggplot2, and while I was unable to replicate the interactivity or the tree-map nature of the graph, the graph still looks pretty cool. Data The data is contained in this pdf file. ... [Read more...]

[Read more...]

[Read more...]

I ended the last post with some pretty plots of air temperature change within and between years in the Central England Temperature series. The elephant in the room1 at the end of that post was is the change in the within year (seasonal) effect over time statistically significant? This is ... [Read more...]

I just noticed that there will be an Introduction to R workshop at the Society for Freshwater Science Annual Meeting in Sacramento on 20-May-20-16. Here is a link to the announcement. [Read more...]

[Read more...]

[Read more...]

guest post by my friend thomas yokota, an oahu-based epidemiologist. palmero professor vito muggeo wrote the joinpoint analysis section of the code below to demonstrate that the segmented package eliminates the need for external (registration-only, windows-only) software. survey package creator and professor thomas lumley wrote the svypredmarg function to replicate ... [Read more...]

I have been happily tweeting the BioMedBridges meeting in Hinxton last week using the #lifesciencedata hashtag, along with more than 100 others, though a small subset was really active. A lot has been published about using Twitter at conference, like the recent paper by Ekins et al (doi:10.1371/journal.pcbi.1003789).The ... [Read more...]

I have been happily tweeting the BioMedBridges meeting in Hinxton last week using the #lifesciencedata hashtag, along with more than 100 others, though a small subset was really active. A lot has been published about using Twitter at conference, like t... [Read more...]

Machine learning is a field of science that focusses on mathematically describing patterns in data. Chemometrics does this for chemical data. Examples are (nano)QSAR where structural information is related to biological activity. I studied during my PhD studies the interaction between the statistics and machine learning with how you ... [Read more...]

Machine learning is a field of science that focusses on mathematically describing patterns in data. Chemometrics does this for chemical data. Examples are (nano)QSAR where structural information is related to biological activity. I studied during my Ph... [Read more...]

[Read more...]

[Read more...]

A question from X validated that took me quite a while to fathom and then the solution suddenly became quite obvious: If a sample taken from an arbitrary distribution on {0,1}⁶ is censored from its (0,0,0,0,0,0) elements, and if the marginal probabilities are know for all six components of the random vector, ... [Read more...]

We have always regretted that we didn’t get to cover gradient boosting in Practical Data Science with R (Manning 2014). To try make up for that we are sharing (for free) our GBM lecture from our (paid) video course Introduction to Data Science. (link, all support material here). Please help ... [Read more...]

Bob Horton Sr Data Scientist, Microsoft Wikipedia describes Simpson’s paradox as “a trend that appears in different groups of data but disappears or reverses when these groups are combined.” Here is the figure from the top of that article (you can click on the image in Wikipedia then follow ... [Read more...]

Microsoft’s newfound desire to make themselves desirable to the hipster development community has caused them to make many things open and/or free of late. One of these manifestations is Visual Studio Code, an Atom-ish editor for us code jockeys. I have friends at Microsoft and the Revolution R ... [Read more...]