You may have seen that we recently open sourced SmartDataCenter and Manta. Because Joyent pioneered the use of infrastructure containers, you would be forgiven for asking the question: does this mean that SDC somehow competes with Docker? The answer is emphatically not — and in fact, to the c… [Read more...]

This post will cover ideas from two individuals: David Varadi of CSS Analytics with whom I am currently collaborating on some … Continue reading → [Read more...]

One of the drawbacks with R has been its limitation with big datasets. It stores everything in RAM so once you have more than 100K records your PC really starts to slow down. However, since AWS allows you to use any size machine, you could now consider using R for ... [Read more...]

We analyzed over six million flights to help you decide on the best time to travel to avoid delays. The post When to fly to get there on time? Six million flights analyzed. appeared first on Decision Science News. [Read more...]

A new release of RcppRedis is now on CRAN. It contains additional commands for hashes and sets, all contributed by John Laing and Whit Armstrong. Changes in version 0.1.2 (2014-11-06) New commands execv, hset, hget, sadd, srem, and smembers... [Read more...]

A couple of weeks ago, Twitter open-sourced their BreakoutDetection package for R, a package designed to determine shifts in time-series data. The Twitter announcement does a great job of explaining the main technique for detection (E-Divisive with Medians), so I won’t rehash that material here. Rather, I wanted to ... [Read more...]

I get questions about this almost every week. Here is an example from a recent comment on this blog: I have two large time series data. One is separated by seconds intervals and the other by minutes. The length of each time series is 180 days. I’m using R (3.1.1) for ... [Read more...]

If you like to make nice looking documents using Latex, I highly recommend using the 'xtable' package. In most instances, it works quite well for producing a reasonable looking table from an R object. I however recently wanted a LaTeX … Continue reading → [Read more...]

Use cases Public reports. Public data sharing, e.g. R packages download logs from CRAN's RStudio mirror - cran-logs.rstudio.com - mask ip addresses. Reports or data sharing for external vendor. Development works can operate on anonymized PRODUCTION data. Manually or semi-manually populated data can often brings some new ... [Read more...]

In my previous blog post, I detailed how I created my first R package called geocodeHERE. This package is a convenient wrapper for Nokia's HERE geocoding API. The cool thing about this API is that it allows for bulk geocoding. So, instead of doing n API calls to geocode n ... [Read more...]

A new release of the stringi package is available on CRAN (please wait a few days for Windows and OS X binary builds). # install.packages("stringi") or update.packages() library("stringi") stringi is an R package providing (but definitely not limiting to) equivalents…Read more › [Read more...]

by Joseph Rickert I recently had the opportunity to look at the data used for the 2009 KDD Cup competition. There are actually two sets of files that are still available from this competition. The "large" file is a series of five .csv files that when concatenated form a data set ... [Read more...]

Today we are announcing that we are open sourcing the two systems at the heart of our business: SmartDataCenter and the Manta object storage platform. SmartDataCenter is the container-based orchestration software that runs the Joyent public cloud; we have used it for the better half of a decade to… [Read more...]

When I start an R class, one of my opening lines is nea […] [Read more...]

Gviz is a really great package for visualising genomics data in R. Recently I have been looking at stranded RNA-seq data, which provides the ability to differentiate sense and antisense expression from a genomic locus thanks to the way in … Continue reading → [Read more...]

I was pleasantly surprised when somebody shared my traveling salesman animation to reddit and the post made it all the way to reddit's default front page (i.e. the top 25). The gif racked up over 1.3 million pageviews on Imgur, a testament to reddit's traffic-generating prowess. Before the post made it ... [Read more...]

A few days ago, I published a post entitled extracting datasets from excel files in a zipped folder, because I wanted to use datasets that were online, in some (zipped) excel format. The first difficult part was the folder with a non-standard character (the French é). Because next week I should ... [Read more...]

On Friday October 31, 2014, and Saturday November 1, 2014, around thirty-five graduate students and faculty members attended a Software Carpentry workshop. Attendees came primarily from the Economics department and the Kellogg School of Management, w... [Read more...]

A minor bugfix release 0.1.1 of the RPushbullet package (interfacing the neat Pushbullet service) landed on CRAN yesterday morning. It cleans up a small issue related to the ability to transfer files between devices via the Pushbullet service where the ability to select a (non-default) target device has now been restored. ... [Read more...]

2006–2014 using FRBData 2006–2014 using FRBData packageTreasury yield curve from the Volcker era through Greenspan, Bernanke, and Yellen. require(YieldCurve) data(FedYieldCurve) maturities I can’t upload a larger view, but just run that code with a bigger width & height, you might get a ~10MB gif. I shrank it further with gifsicle. ... [Read more...]