New package: GetQuandlData

September 30, 2019
By

[This article was first published on R on msperlin, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Introduction

Quandl is one of the best platforms for finding and downloading financial and economic time series. The collection of free databases is solid and I’ve used it intensively in my research and class material.

But, a couple of things from the native package Quandl always bothered me:

  • Multiple data is always returned in the wide (column oriented) format (why??);
  • No local caching of data;
  • No control for importing error and status;
  • Not easy to work within the tidyverse collection of packages

As you suspect, I decided to tackle the problem over the weekend. The result is package GetQuandlData. This is what it does differently:

  • It uses the json api (and not the Quandl native function), so that some metadata is also returned;
  • The resulting dataframe is always returned in the long format, even for multiple series;
  • Users can set custom names for input series. This is very useful when using along ggplot or making tables;
  • Uses package memoise to set a local caching system. This means that the second time you ask for a particular time series, it will grab it from your hard drive (and not the internet);
  • Always compares the requested dates against dates available in the platform.

Installation

# not in CRAN yet (need to test it further)
#install.packages('GetQuandlData')

# from github
devtools::install_github('msperlin/GetQuandlData')

Example 01 – Inflation in the US

Let’s download and plot information about inflation in the US:

library(GetQuandlData)
library(tidyverse)

my_id <- c('Inflation USA' = 'RATEINF/INFLATION_USA')
my_api <- readLines('~/Dropbox/.quandl_api.txt') # you need your own API (get it at https://www.quandl.com/sign-up-modal?defaultModal=showSignUp>)
first_date <- '2000-01-01'
last_date <- Sys.Date()

df <- get_Quandl_series(id_in = my_id, 
                        api_key = my_api, 
                        first_date = first_date,
                        last_date = last_date, 
                        cache_folder = tempdir())

glimpse(df)
## Observations: 236
## Variables: 4
## $ series_name  "Inflation USA", "Inflation USA", "Inflation USA", "…
## $ ref_date     2019-08-31, 2019-07-31, 2019-06-30, 2019-05-31, 201…
## $ value        1.750, 1.811, 1.648, 1.790, 1.996, 1.863, 1.520, 1.5…
## $ id_quandl    "RATEINF/INFLATION_USA", "RATEINF/INFLATION_USA", "R…

As you can see, the data is in the long format. Let’s plot it:

p <- ggplot(df, aes(x = ref_date, y = value/100)) + 
  geom_col() + 
  labs(y = 'Inflation (%)', 
       x = '',
       title = 'Inflation in the US') + 
  scale_y_continuous(labels = scales::percent)

p

Beautiful!

Example 02 – Inflation for many countries

Next, lets have a look into a more realistic case, where we need inflation data for several countries:

First, we need to see what are the available datasets from database RATEINF:

library(GetQuandlData)
library(tidyverse)

db_id <- 'RATEINF'
my_api <- readLines('~/Dropbox/.quandl_api.txt') # you need your own API

df <- get_database_info(db_id, my_api)

knitr::kable(df)
code name description refreshed_at from_date to_date quandl_code quandl_db
CPI_ARG Consumer Price Index – Argentina Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-09-28 02:19:58 1988-01-31 2013-12-31 RATEINF/CPI_ARG RATEINF
CPI_AUS Consumer Price Index – Australia Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-09-28 02:19:59 1948-09-30 2019-06-30 RATEINF/CPI_AUS RATEINF
CPI_CAN Consumer Price Index – Canada Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-09-28 02:19:59 1989-01-31 2019-08-31 RATEINF/CPI_CAN RATEINF
CPI_CHE Consumer Price Index – Switzerland Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-09-28 02:19:58 1983-01-31 2019-08-31 RATEINF/CPI_CHE RATEINF
CPI_DEU Consumer Price Index – Germany Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-09-28 02:19:59 1991-01-31 2019-08-31 RATEINF/CPI_DEU RATEINF
CPI_EUR Consumer Price Index – Euro Area Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-09-28 02:19:59 1990-01-31 2019-08-31 RATEINF/CPI_EUR RATEINF
CPI_FRA Consumer Price Index – France Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-09-28 02:19:59 1990-01-31 2019-08-31 RATEINF/CPI_FRA RATEINF
CPI_GBR Consumer Price Index – UK Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-09-28 02:19:58 1988-01-31 2019-08-31 RATEINF/CPI_GBR RATEINF
CPI_ITA Consumer Price Index – Italy Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-09-28 02:19:59 2001-01-31 2019-08-31 RATEINF/CPI_ITA RATEINF
CPI_JPN Consumer Price Index – Japan Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-09-28 02:19:59 1970-01-31 2019-08-31 RATEINF/CPI_JPN RATEINF
CPI_NZL Consumer Price Index – New Zealand Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-09-28 02:19:59 1988-03-31 2019-06-30 RATEINF/CPI_NZL RATEINF
CPI_RUS Consumer Price Index – Russia Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-09-28 02:19:59 1995-01-31 2019-07-31 RATEINF/CPI_RUS RATEINF
CPI_USA Consumer Price Index – USA Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-09-28 02:19:59 1913-01-31 2019-08-31 RATEINF/CPI_USA RATEINF
INFLATION_ARG Inflation YOY – Argentina Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-09-28 02:19:58 1989-01-31 2013-12-31 RATEINF/INFLATION_ARG RATEINF
INFLATION_AUS Inflation YOY – Australia Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-09-28 02:19:59 1949-03-31 2019-06-30 RATEINF/INFLATION_AUS RATEINF
INFLATION_CAN Inflation YOY – Canada Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-09-28 02:19:59 1990-01-31 2019-08-31 RATEINF/INFLATION_CAN RATEINF
INFLATION_CHE Inflation YOY – Switzerland Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-09-28 02:19:59 1984-01-31 2019-08-31 RATEINF/INFLATION_CHE RATEINF
INFLATION_DEU Inflation YOY – Germany Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-09-28 02:19:59 1992-01-31 2019-08-31 RATEINF/INFLATION_DEU RATEINF
INFLATION_EUR Inflation YOY – Euro Area Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-09-28 02:19:59 1991-01-31 2019-08-31 RATEINF/INFLATION_EUR RATEINF
INFLATION_FRA Inflation YOY – France Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-09-28 02:19:59 1991-01-31 2019-08-31 RATEINF/INFLATION_FRA RATEINF
INFLATION_GBR Inflation YOY – UK Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-09-28 02:19:59 1989-01-31 2019-08-31 RATEINF/INFLATION_GBR RATEINF
INFLATION_ITA Inflation YOY – Italy Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-09-28 02:19:59 2002-01-31 2019-08-31 RATEINF/INFLATION_ITA RATEINF
INFLATION_JPN Inflation YOY – Japan Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-09-28 02:19:59 1971-01-31 2019-08-31 RATEINF/INFLATION_JPN RATEINF
INFLATION_NZL Inflation YOY – New Zealand Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-09-28 02:19:59 2001-03-31 2019-06-30 RATEINF/INFLATION_NZL RATEINF
INFLATION_RUS Inflation YOY – Russia Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-09-28 02:19:59 1996-01-31 2019-07-31 RATEINF/INFLATION_RUS RATEINF
INFLATION_USA Inflation YOY – USA Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2019-09-28 02:19:59 1914-01-31 2019-08-31 RATEINF/INFLATION_USA RATEINF

Nice. Now we only need to filter the series with YOY inflation:

idx <- stringr::str_detect(df$name, 'Inflation YOY')
df_series <- df[idx, ]

and grab the data:

my_id <- df_series$quandl_code
names(my_id) <- df_series$name
first_date <- '2010-01-01'
last_date <- Sys.Date()

df_inflation <- get_Quandl_series(id_in = my_id, 
                                  api_key = my_api,
                                  first_date = first_date,
                                  last_date = last_date)

glimpse(df_inflation)
## Observations: 897
## Variables: 4
## $ series_name  "Inflation YOY - Argentina", "Inflation YOY - Argent…
## $ ref_date     2013-12-31, 2013-11-30, 2013-10-31, 2013-09-30, 201…
## $ value        10.95, 10.54, 10.55, 10.49, 10.55, 10.61, 10.46, 10.…
## $ id_quandl    "RATEINF/INFLATION_ARG", "RATEINF/INFLATION_ARG", "R…

And, finally, an elegant plot:

p <- ggplot(df_inflation, aes(x = ref_date, y = value/100)) + 
  geom_col() + 
  labs(y = 'Inflation (%)', 
       x = '',
       title = 'Inflation in the World',
       subtitle = paste0(first_date, ' to ', last_date)) + 
  scale_y_continuous(labels = scales::percent) + 
  facet_wrap(~series_name)

p

To leave a comment for the author, please follow the link and comment on their blog: R on msperlin.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.



If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Comments are closed.

Search R-bloggers

Sponsors

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)