Evidence for a limit to effective peer review

December 18, 2016

(This article was first published on R – What You're Doing Is Rather Desperate, and kindly contributed to R-bloggers)

I missed it first time around but apparently, back in October, Nature published a somewhat-controversial article: Evidence for a limit to human lifespan. It came to my attention in a recent tweet:

The source: a fact-check article from Dutch news organisation NRC titled “Nature article is wrong about 115 year limit on human lifespan“. NRC seem rather interested in this research article. They have published another more recent critique of the work, titled “Statistical problems, but not enough to warrant a rejection” and a discussion of that critique, Peer review post-mortem: how a flawed aging study was published in Nature.

Unfortunately, the first NRC article does itself no favours by using non-comparable x-axis scales for its charts and not really explaining very well how the different datasets (IDL and GRG) were used. Data nerds everywhere then, are wondering whether to repeat the analysis themselves and perhaps fire off a letter to Nature.

My advice: don’t waste your time since Philipp Berens and Tom Wallis already did a great job, which is described here and documented (in a manner that you wish applied to Nature articles) at Github.

Maximum reported age at death (n = 33) showing fitted segmented regression

Maximum reported age at death (n = 33) showing fitted segmented regression

That said: I have wasted a little time and the results are in this Github repository. My code illustrates one way to obtain the data, and apply some regression models (linear, segmented and LOESS). One contentious aspect of the original paper was the choice of breakpoint (1995) for segmented regression, which seemed rather arbitrary. In fact it is possible to fit something very similar (albeit with a breakpoint in 1999) to the “maximum age at death by year” data using the R segmented package. Thanks to Nathan Lemoine for his tutorial on that topic.

As to whether segmented linear regression should have been applied, the arguments continue. According to Berens and Wallis, it is supported by the GRG dataset but not by the data from the IDL. All of the articles linked to in this post are worth a read, in particular the third of the NRC articles which contains some telling quotes regarding how articles come to be published. This one is my favourite:

Until then, I had thought focusing on statistics was a major task for a reviewer. But at Nature, that is apparently not a problem at all. In the eleven primary questions drawn up by the journal for an “ideal review”, statistics and methodology are not explicitly addressed. They feature in the second list of questions, for “if time is available”.

Filed under: publications, R, statistics Tagged: human, longevity, peer review

To leave a comment for the author, please follow the link and comment on their blog: R – What You're Doing Is Rather Desperate.

R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...

If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Comments are closed.


Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)