by Katherine Zhao, Hong Lu, Zhongmou Li, Data Scientists at Microsoft Bicycle rental has become popular as a convenient and environmentally friendly transportation option. Accurate estimation of bike demand at different locations and different times would help bicycle-sharing systems better meet rental demand and allocate bikes to locations. In this ... [Read more...]

by Yuzhou Song, Microsoft Data Scientist R is an open source, statistical programming language with millions of users in its community. However, a well-known weakness of R is that it is both single threaded and memory bound, which limits its ability to process big data. With Microsoft R Server (MRS), ... [Read more...]

by Joseph Rickert When I first went to grad school, the mathematicians advised me cultivate the habit of reading with a pencil. This turned into a lifelong habit and useful skill for reading all sorts of things: literature, reports and newspapers for example; not just technical papers. However, reading statistics ... [Read more...]

by Joseph Rickert Answering email queries from friends and acquaintances from around the world wanting to attend useR! 2016 has been painful. It is amazing that the conference sold out a full two months before its start, but upon reflection, not unbelievable. From its inception useR! has been an "academic" conference ... [Read more...]

by Lixun Zhang, Data Scientist at Microsoft As a data scientist, I have experience with R. Naturally, when I was first exposed to Microsoft R Open (MRO, formerly Revolution R Open) and Microsoft R Server (MRS, formerly Revolution R Enterprise), I wanted to know the answers for 3 questions: What do ... [Read more...]

by Joseph Rickert R/Finance 2016 is less than a month away and, as always, I am very much looking forward to it. In past years, I have elaborated on what puts it among my favorite conferences even though I am not a finance guy. R/Finance is small, single track ... [Read more...]

by Joseph Rickert The R Consortium, the non-profit trade organization formed under the Linux Foundation to support the R language and the R Community, is beginning to build real momentum. First of all, two new companies recently joined the Consortium: Avant which provides online personal and auto loans and Procogia, ... [Read more...]

by Joseph Rickert Basically, there are two kinds of graphics or plots you can make from a data set: (1) those that allow you to see what is going on with the data, and (2) those you make to communicate what you have found to someone else. When making the first kind, ... [Read more...]

By Srini Kumar, Director of Data Science at Microsoft Who does not hate being stopped and given a traffic ticket? Invariably, we think that something is not fair that we got it and everyone else did not. I am no different, and living in the SF Bay Area, I have ... [Read more...]

by Joseph Rickert Packages continue to flood into CRAN at a rate the challenges the sanity of anyone trying to keep up with what's new. So far this month, more than 190 packages have been added. Here is a my view of what's interesting in this March madness. The launch_tutorial() ... [Read more...]

by Bob Horton, Senior Data Scientist, Microsoft This is a follow-up to my earlier post on learning curves. A learning curve is a plot of predictive error for training and validation sets over a range of training set sizes. Here we’re using simulated data to explore some fundamental relationships ... [Read more...]

by Joseph Rickert Last year, I wrote the New York R Conference “set the bar pretty darn high for a first time conference”. Not only was there an outstanding lineup of speakers, but the energy and enthusiasm that conference attendees brought with them, or maybe just generated on the spot, ... [Read more...]

by Joseph Rickert In a post late last year, my colleague and fellow blogger, Andrie de Vries described enhancements to the AzureML R package that makes it easy to publish R functions that consume data frames as Azure Web Services. A very nice consequence is that it is now feasible ... [Read more...]

by Said Bleik, Shaheen Gauher, Data Scientists at Microsoft Evaluation metrics are the key to understanding how your classification model performs when applied to a test dataset. In what follows, we present a tutorial on how to compute common metrics that are often used in evaluation, in addition to metrics ... [Read more...]

by Joseph Rickert I always think of Strata Hadoop World and Predictive Analytics World as initiating the Spring conference season here in the San Francisco Bay Area. The rainy season is usually over by the end of March and it is a perfect time to visit. If you are traveling ... [Read more...]

by Joseph Rickert Random Forests, the "go to" classifier for many data scientists, is a fairly complex algorithm with many moving parts that introduces randomness at different levels. Understanding exactly how the algorithm operates requires some work, and assessing how good a Random Forests model fits the data is a ... [Read more...]

by Verena Haunschmid Since I have a cat tracker, I wanted to do some analysis of the behavior of my cats. I have shown how to do some of these things here. Data Collection The data was collected using the Tractive GPS Pet Tracker over a period of about one ... [Read more...]

By Joseph Rickert The ability to generate synthetic data with a specified correlation structure is essential to modeling work. As you might expect, R’s toolbox of packages and functions for generating and visualizing data from multivariate distributions is impressive. The basic function for generating multivariate normal data is mvrnorm() ... [Read more...]

by Daniel Moore Director of Applied Statistics Engineering, Console Development Microsoft In Xbox Hardware, we are interested in the various ways that our hardware is used, and we are especially interested in how that usage changes over time. We employ several several time series analysis techniques that are helpful in ... [Read more...]

by Joseph Rickert Earlier this month the Bay Area useR Group (BARUG) held it annual lightning talk meeting. This is by far our most popular meeting format: eight, 15 minute talks (12 minutes speaking and 3 minutes Q & A while the next speaker is setting up) packed into a two hour time slot. ... [Read more...]