Introducing FlyVis

[This article was first published on Frank Portman, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

After a couple all-nighters we’re finally done with our undergraduate statistics thesis. The abstract provides a brief overview of what we were trying to accomplish:

We explore the possibility of improving data analysis through the use of interactive visualization.
Exploration of data and models is an iterative process. We hypothesize that dynamic, interactive visualizations
greatly reduce the cost of new iterations and thus f acilitate agile investigation and rapid prototyping. Our
web-application framework,, offers evidence for such a hypothesis for a dataset consisting of airline
on-tim e flight performance between 2006-2008. Utilizing our framework we are able to study the feasibility of modeling
subsets of flight delays from temporal data, which fails on the full dataset.

Technically, this was a very fun project. Shiny is an extremely powerful package which provides the interactive framework necessary to build such applications. We also made use of the JavaScript library leaflet.js for the interactive map. All in all, I learned quite a bit about writing efficient R code, as the dataset we were using had over 18 million observations.

To learn more about the app check out the projects page or the actual application website

FlyVis lets you dynamically explore the airports on-time dataset which yields some pretty interesting graphs. For example, if we look at the intraday distribution of flights and delays for Memphis:

we see a pretty interesting pattern. Turns out the FedEx shipments control most of the flights out of Memphis which gives us this unique shape.

The beauty of an interactive application is that you (the reader) can discover something that I haven’t even considered. I merely provide the tools and you can explore. If anybody finds some cool patterns in certain airports then I’d love to hear about it over e-mail or comment.

Once we polish the application a bit more we will release the source code on GitHub. Disclaimer: The site will initially take a minute or two to load since our server has to load the massive dataset into memory. Also, the plots do take 5-6 seconds to generate. Again this is due to the size of our data and is something we are currently trying to optimize.

Finally, what would this post be without some R code? Here’s what we used for the Calendar Heatmap plot:

<span class="line">myCalPlot <span class="o"><-</span> <span class="kr">function</span> <span class="p">(</span>dates<span class="p">,</span> values<span class="p">)</span>
</span><span class="line"><span class="p">{</span>
</span><span class="line">  <span class="kr">if</span> <span class="p">(</span><span class="o">!</span><span class="kn">require</span><span class="p">(</span>ggplot2<span class="p">)</span> <span class="o">&</span> <span class="o">!</span><span class="kn">require</span><span class="p">(</span>plyr<span class="p">))</span>
</span><span class="line">    <span class="kp">stop</span><span class="p">(</span><span class="s">"The packages ggplot2 and plyr are required to use plotCalendarHeatmap"</span><span class="p">)</span>
</span><span class="line">  tp <span class="o">=</span> projectDate<span class="p">(</span>dates<span class="p">,</span> drop <span class="o">=</span> <span class="bp">F</span><span class="p">)</span>
</span><span class="line">  tp<span class="o">$</span>values <span class="o">=</span> values
</span><span class="line">  tp<span class="o">$</span>week <span class="o">=</span> <span class="kp">as.numeric</span><span class="p">(</span><span class="kp">format</span><span class="p">(</span>dates<span class="p">,</span> <span class="s">"%W"</span><span class="p">))</span>
</span><span class="line">  tp<span class="o">$</span>month <span class="o"><-</span> <span class="kp">factor</span><span class="p">(</span>tp<span class="o">$</span>month<span class="p">,</span>levels<span class="o">=</span><span class="kp">as.character</span><span class="p">(</span><span class="m">1</span><span class="o">:</span><span class="m">12</span><span class="p">),</span>
</span><span class="line">                     labels<span class="o">=</span><span class="kt">c</span><span class="p">(</span><span class="s">"Jan"</span><span class="p">,</span><span class="s">"Feb"</span><span class="p">,</span><span class="s">"Mar"</span><span class="p">,</span><span class="s">"Apr"</span><span class="p">,</span><span class="s">"May"</span><span class="p">,</span><span class="s">"Jun"</span><span class="p">,</span>
</span><span class="line">                              <span class="s">"Jul"</span><span class="p">,</span><span class="s">"Aug"</span><span class="p">,</span><span class="s">"Sep"</span><span class="p">,</span><span class="s">"Oct"</span><span class="p">,</span><span class="s">"Nov"</span><span class="p">,</span><span class="s">"Dec"</span><span class="p">),</span>ordered<span class="o">=</span><span class="kc">TRUE</span><span class="p">)</span>
</span><span class="line">
</span><span class="line">  tp <span class="o">=</span> ddply<span class="p">(</span>tp<span class="p">,</span> <span class="m">.</span><span class="p">(</span>year<span class="p">,</span> month<span class="p">),</span> <span class="kp">transform</span><span class="p">,</span> monthweek <span class="o">=</span> <span class="m">1</span> <span class="o">+</span>
</span><span class="line">               week <span class="o">-</span> <span class="kp">min</span><span class="p">(</span>week<span class="p">))</span>
</span><span class="line">  ggplot<span class="p">(</span>tp<span class="p">,</span> aes<span class="p">(</span>monthweek<span class="p">,</span> weekday<span class="p">,</span> fill <span class="o">=</span> values<span class="p">))</span> <span class="o">+</span> geom_tile<span class="p">(</span>colour <span class="o">=</span> <span class="s">"white"</span><span class="p">)</span> <span class="o">+</span>
</span><span class="line">    facet_grid<span class="p">(</span><span class="m">.</span> <span class="o">~</span> month<span class="p">)</span> <span class="o">+</span>
</span><span class="line">    scale_fill_gradientn<span class="p">(</span>colours <span class="o">=</span> <span class="kp">rev</span><span class="p">(</span><span class="kt">c</span><span class="p">(</span><span class="s">"#D61818"</span><span class="p">,</span> <span class="s">"#FFAE63"</span><span class="p">,</span> <span class="s">"#FFFFBD"</span><span class="p">,</span> <span class="s">"#B5E384"</span><span class="p">)))</span> <span class="o">+</span> theme_bw<span class="p">()</span>
</span><span class="line"><span class="p">}</span>

Doesn’t include the pre-processing but that’ll be out early 2014.

To leave a comment for the author, please follow the link and comment on their blog: Frank Portman. offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)