I’ve received a few emails about including regression variables (i.e., covariates) in TBATS models. As TBATS models are related to ETS models, tbats() is unlikely to ever include covariates as explained here. It won’t actually complain if you include an xreg argument, but it will ignore it. ... [Read more...]

Monte Carlo simulation and resampling methods for social science is a short paperback written by Thomas Carsey and Jeffrey Harden on the use of Monte Carlo simulation to evaluate the adequacy of a model and the impact of assumptions behind this model. I picked it in the library the other ... [Read more...]

My friend and colleague Andreas sent me a link to a working paper published by a statistician at the University of Christchurch (New Zealand) and discussed here. The main idea of the paper was to use a Bayesian model to predict the number of futur... [Read more...]

This is my second post in converting SAS's PROC MCMC examples in R. The task in his week is determining the transformation parameter in a Box-Cox transformation. SAS only determines Lambda, but I am not so sure about that. What I used to do was get an ... [Read more...]

Similar to the row search, by-group aggregation is another perfect use case to demonstrate the power of split-and-conquer with parallelism. In the example below, it is shown that the homebrew by-group aggregation with foreach pakage, albeit inefficiently coded, is still a lot faster than the summarize() function in Hmisc package. [Read more...]

What is a Prediction Problem?A business problem which involves predicting future events by extracting patterns in the historical data. Prediction problems are solved using Statistical techniques, mathematical models or machine learning techniques.For example: Forecasting stock price for the next week, predicting which football team wins the world cup, ... [Read more...]

Last week, @dmonniaux published an interesting post entitled l’erreur n’a rien d’original  on  his blog. He was asking the following question : let , and denote three real-valued coefficients, under which assumption on those three coefficients does has a real-valued root ? Everyone aswered , but no one mentioned that it ... [Read more...]

When I am am working in Sql Server and need to send an email I use "sp_send_dbmail". So when I am working in R, I didn't know how to send an email. I often use this as notification that a … Continue reading → [Read more...]

by Andrie deVries One of the reasons that R is so popular is the CRAN archive of useful packages. However, with more than 5,900 packages on CRAN, many organisations need to maintain a private mirror of CRAN with only a subset of packages that are relevant to them. The package miniCRAN ... [Read more...]

Anders Ekholm has written a wonderful paper Ekholm, Anders G. Components of Portfolio Variance: Systematic, Selection and Timing August 8, 2014 http://ssrn.com/abstract=2463649 demonstrating how we might decompose a money manager’s performance with... [Read more...]

A month ago when the WHO was projecting estimates of the effect of current outbreak of Ebola being as deadly as affecting 20,000 people, I ran some elementary modelling and found that these estimates are far too small given the current trend.  The... [Read more...]

Return on Investment (ROI) is management's bottom line. Consequently, everything must be separated and assigned a row with associated costs and profits. Will we make more by adding another product to our line? Will we lose sales by limiting the feature... [Read more...]

  I frequently find myself thinking of the best way to convince an SPSS user to make the switch to R. In the process, I came up with the three following most common objections by SPSS users and my responses.   Objection #1) I can’t use R because I don’t know ... [Read more...]

SupStat Analytics and Transwarp Technologies will be at the 2014 Strata Conference and Hadoop World showcasing the power of Hadoop and Spark computing with R analytics. We’re excited to be presenting to the data science world the Transwarp Data Hub, an integrated storage, processing, and analytics platform that delivers up ... [Read more...]

One of the things that I wondered about regarding the previous post was how would this strategy have performed in … Continue reading → [Read more...]

[Read more...]

by Joseph Rickert Recently, I had the opportunity to present a webinar on R and Data Science. The challenge with attempting this sort of thing is to say something interesting that does justice to the subject while being suitable for an audience that may include both experienced R users and ... [Read more...]

Four of DC2′s board members have published a new book! Tony Ojeda, Sean Murphy, Benjamin Bengfort, and Abhijit Dasgupta are proud to announce the arrival of Practical Data Science Cookbook (Packt, $10 ebook or $49.99 print+ebook). Practical Data Science Cookbook is perfect for … Continue reading → The post Announcing the Publication ... [Read more...]

This is part two of a three-part blog series on building a DGA classifier and it is split into the three phases of building a classifier: 1) Data preperation 2) Feature engineering and 3) Model selection. Back in part 1, we prepared the data and we are starting with a nice clean list of ... [Read more...]

Say your function in sapply returns a matrix with a variable number of rows. For example sapply is too stupid to see the pattern here (or maybe I don’t know how to cast the return value into an appropriate shape…). The result of the above is a list: [[1]] [,1] [,2] [,3] [1,] 1 1 […] [Read more...]