The Financial Crisis on Tape Part II

[This article was first published on Joe's Data Diner, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

EDIT: I am aware of some browsers failing to load the github code below. I will try to improve this as soon as possible. Until then it may work better on than at Finally all the code is available at github.

First, I would like to welcome all of you reading this via R-Bloggers. I have been a user of this superb resource for years and I’m proud that I now have the opportunity to contribute!

In The Financial Crisis on Tape Part I I demonstrated how inter-sector and inter-asset correlation changed during the Financial Crisis by using a heat map to visualise such correlations. Today, I will use these heat maps as individual frames to create a video. I hope the video will provide a nice trailer to the Financial Crisis if not completely catching it on Tape.

As usual, some of my readers may be more interested in the finance-side than the R-side of this post (although I hope to inspire as much migration from excel based analytics as possible). Therefore, I like to provide the results at the start. I know it may not be as humorous as some you tube clips, but it is only 40 seconds long. So, enjoy!

The following assets are shown in the video:
SPY: S&P 500
QQQ: Nasdaq 100
EEM: Emerging Markets
IWM: Russel 2000
EFA: EAFE (Europe, Australasia and Far East)
TLT: 20 Year Treasury
IYR: U.S. Real Estate

The chart on the right is hopefully familiar enough; it is a heat map showing the correlation between a number of assets. As the video progresses, the six month calculation period for the correlations moves forward in time. We start with a heat map of the correlation computed over the first 6 months of 2005 and end with the heat map of the correlation computed over the most recent 6 months of data. The changes in the correlation structure – especially the increases in correlation during periods of stress – should be evident.

The graph on the left is intended to provide the context, as some of us may not recall the exact market movements of the assets since 2005. The grey box highlights the six month period of each asset’s time series which was used to compute the correlations shown in the heat map to the right. This should make it easier to link the changes in the correlation to market conditions. The coloured dots represent the last day of this backward looking window.

If you’re still reading, I guess you might like to know how the video was made? You create all the individual frames as png files using R. Then you combine the frames into an mpeg using ffmeg (windows download). Nothing easier than that?!
It’s really not as complicated as it sounds. For more detail, please refer to this previous R-Blog – thanks Civil Statistician. The animation package could provide a nice introduction too. But now let’s take a walk through the R-code to see how to make the individual png files.

First, we need to load the data. I have used the QuantMod package and I’d like to thank Systematic Investor for his useful posts on the subject.

Next, we compute the historical returns. These historical returns will be used to compute the correlations. We also need to keep the price time series to produce the graph on the left of the video. Finally we massage the results a little to make the calculations a easier and get the assets plotted in the desired order.

Now we have all the data we need to compute the rolling-correlations. The comments within the code provide details for each step.

Then, the fun part: plotting! This relies heavily on ggplot2. Perhaps unsurprisingly PlotCorrelsForThisDate takes a date as its argument, looks up the correlation data (which we obtained earlier) and plots a heat map.

We create the price time series graph with the ProduceFilmStill and plot it next to the correlation heat map using the gtable package to line everything up nicely. Finally, we save the plot as a png in a directory. Using numbers rather than dates to name the files makes using ffmpeg super-easy because the argument of ProduceFilmStill is an integer referencing a position in and not the date itself.

To finish, we run this function for all the dates for which we have data and use the ffmpeg command shown (at the command line) to create the mpeg. Done!

And well done for making it this far; I hope you enjoyed it. As I’m still new to blogging, any questions or comments, especially suggestions how to improve my posts, are much appreciated!

For further reading, here’s the link to the static version of today’s video from The Financial Crisis on Tape Part I. It’s worth a read if you haven’t seen it yet. While it doesn’t show the changes in correlation with the same granularity as the video, it might be easier to compare disparate periods of time when one can see them all at once. You be the judge!

To leave a comment for the author, please follow the link and comment on their blog: Joe's Data Diner. offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)