[Read more...]

Note: the title of this post was inspired by this question on stackoverflow. This section gives the basic facts and recommendations for importing files with arbitrary encoding on Windows. The issues described here by and large to not apply on Mac o... [Read more...]

by Dmitry Pechyoni, Microsoft Data Scientist The New York City taxi dataset is one of the largest publicly available datasets. It has about 1.1 billion taxi rides in New York City. Previously this dataset was explored and visualized in a number of blog posts, where the authors used various technologies (e.... [Read more...]

1. OverviewA growing number of R packages are created by various people in the world. A part of the cause of it is the devtools package that makes it easy to develop R packages [1]. The devtools package not only facilitates the process to develop R packages but also provides an another ... [Read more...]

Most of the time between starting manuscript and having it accepted after peer-review is spent writing, re-writing and re-arranging content. In Word, keeping track of figure numbers is a big pain, even more so when figures are moved between the main manuscript and the supplement. Moving my Word manuscript to ... [Read more...]

Most of the time between starting manuscript and having it accepted after peer-review is spent writing, re-writing and re-arranging content. In Word, keeping track of figure numbers is a big pain, even more so when figures are moved between the main ma... [Read more...]

Have you ever suffered because of the impossibility of reproducing graphs, tables or analysis’ results in R? Have you ever bothered yourself for not being able to share R objects (i.e., plots or final analysis models) within your reports, posters or articles? Or maybe simply you have too many ... [Read more...]

As I was preparing for an R intro course I came up with the idea of creating a fake data set that is stuffed full of all the conceivable errors one can imagine. Just in case my imagination falls short, I’d appreciate all the suggestions in the comments so ... [Read more...]

How to make a radial plot without using radial coordinates with plotly. Let's load a few libraries and pull some sample data. Then do some preparation by adding data associated with turning the graphic into a circle. A trivial application of high school SOHCATTOA, and finally plot. Comments included at [Read more...]

[Read more...]

The open-source R language is the most frequently used analytics / data science software, selected by 49% of the 2895 voters of the 2016 KDNuggets Software Poll. (R was also the top selection in last year's poll.) Python was a close second at 45.8%, and SQL was third at 35.5%. (Respondents could select multiple tools in ... [Read more...]

EARL is a Conference for users and developers of the open source R programming language. The primary focus of the Conference is the commercial usage of R across a range of industry sectors with the aim of sharing knowledge and … Continue reading → [Read more...]

Last week I gave a seminar for around 40 analysts from another government agency on using graphics to represent data. In doing such presentations, I usually focus on different purposes of graphics: exploratory as part of the analysis workflow (eg as diagnosis for statistical models) for presenting results Exactly what the ... [Read more...]

Introduction Today we’ll see what happens when you have not one, but two variables in your model. We will also continue to use some old and new dplyr calls, as well as another parameter for our ggplot2 figure. I’ll be taking for granted some of the set-up steps ... [Read more...]

[Read more...]

Introduction The new R package, manhattanly, creates interactive manhattan plots using the plotly.js engine. The plots are usable from the R console, the RStudio viewer pane, R Markdown documents, in Shiny apps, embeddable in websites and can be exported as .png files. By hovering the mouse over a point, ... [Read more...]

THIS POST IS NO LONGER ENTIRELY RELEVANT. DPLYR 0.7 HAS A SLIGHTLY DIFFERENT (AND SLIGHTLY MORE INTUITIVE) WAY OF WORKING WITH NON-STANDARD EVALUATION. I love the dplyr package with all of its functions, however if you use normal dplyr in functions in your package r-cmd-check will give you a warning: R ... [Read more...]

I love the dplyr package with all of its functions, however if you use normal dplyr in functions in your package r-cmd-check will give you a warning: R CMD check NOTE: No visible binding for global variable NAME OF YOUR VARIABLE 1. The functions do work, and everything is normal, however ... [Read more...]

While developing risk models with hundreds of potential variables, we often run into the situation that risk characteristics or macro-economic indicators are highly correlated, namely multicollinearity. In such cases, we might have to drop variables with high VIFs or employ “variable shrinkage” methods, e.g. lasso or ridge, to suppress ... [Read more...]