Model results can be thought of as a reward for the many hours of model design, troubleshooting, re-design, etc. that analyses often require. Following the potentially exhausting mental exercise to acquire these results, I think we’d all like the interpretation … Continue reading →

Demonstrating the various ways percentile calculations can be undertaken in R and specifically with respect to measuring turbidity in water supplies. Continue reading → The post Percentile Calculations in Water Quality Regulations appeared first on The Devil is in the Data.

Factor analysis is a controversial technique that represents the variables of a dataset as linearly related to random, unobservable variables called factors, denoted where . The factors are representative of ‘latent variables’ underlying the original variables. The existence of the factors is hypothetical as they cannot be measured or observed.... The ...

In this post we are going to have a look at one of the problems while applying clustering algorithms such as k-means and expectation maximization that is of determining the optimal number of clusters. The problem of determining what will be the best value for the number of clusters is ...

If you do enough web scraping, you’ll eventually hit a wall that the trusty httr verbs (that sit beneath rvest) cannot really overcome: dynamically created content (via javascript) on a site. If the site was nice enough to use XHR requests to load the dynamic content, you can generally ...

Solution to Euler Problem 11 in the R language. What is the greatest product of four adjacent numbers in the same direction in the 20 by 20 grid? Continue reading → The post Euler Problem 11: Largest Product in a Grid appeared first on The Devil is in the Data.

Introduction Malaysia is the leading producer of natural rubber in the world. Being a leader in the production of natural rubber, Malaysia is contributing around 46% of total rubber production in the world. The rubber plantation was started in Malaysi...

We will use Document-Term Matrix that is the result of Vocabulary-based vectorization for training the model for Twitter sentiment analysisRecently I’ve worked with word2vec and doc2vec algorithms that I found interesting from many perspectives. Even though I used them for another purpose, the main thing they were ...

Recently I’ve worked with word2vec and doc2vec algorithms that I found interesting from many perspectives. Even though I used them for another purpose, the main thing they were developed for is Text analysis. As I noticed, my 2014 year’s article Twitter sentiment analysis is one of the ...