It has been a while since I discussed testing for overfitting in backtests. Since then, Marcos López de Prado and coauthors have done some very thoughtful work (see the bottom), and they even started a blog. Their newest paper builds on discoveries they made in their earlier work, and is an absolute must-read.
Bailey, David H. and Borwein, Jonathan M. and Lopez de Prado, Marcos and Zhu, Qiji Jim
Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance (October 7, 2013)
Available at SSRN: http://ssrn.com/abstract=2308659
Translating scientific papers into code is not always easy, but I spent some time implementing some of the concepts in R, so that I can understand this more fully. Just as a word of encouragement to others out there, I am no math genius nor have any advanced math education, so please don’t be intimidated by formulas. Below you will see a slidify/rCharts discussion demonstrating these first steps. I plan to research this much more thoroughly. As always, I blog to interact, so please let me know what you are thinking.
R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...