Access the free economic database DBnomics with R

March 17, 2019
By

[This article was first published on Macroeconomic Observatory - R, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

DBnomics : the world’s economic database

Explore all the economic data from different providers (national and international statistical institutes, central banks, etc.), for free, following the link db.nomics.world.

You can also retrieve all the economic data through the rdbnomics package here. This blog post describes the different ways to do so.

Fetch time series by ids

First, let’s assume that we know which series we want to download. A series identifier (ids) is defined by three values, formatted like this: provider_code/dataset_code/series_code.

Fetch one series from dataset ‘Unemployment rate’ (ZUTN) of AMECO provider

library(magrittr)
library(dplyr)
library(ggplot2)
library(rdbnomics)
df <- rdb(ids = 'AMECO/ZUTN/EA19.1.0.0.0.ZUTN') %>%
  filter(!is.na(value))

In such data.frame, you will always find at least nine columns:

  • provider_code
  • dataset_code
  • dataset_name
  • series_code
  • series_name
  • original_period (a character string)
  • period (a date of the first day of original_period)
  • value
  • @frequency (harmonized frequency generated by DBnomics)

The other columns depend on the provider and on the dataset. They always come in pairs (for the code and the name). In the data.frame df, you have:

  • unit (code) and Unit (name)
  • geo (code) and Country (name)
  • freq (code) and Frequency (name)

plot of chunk unnamed-chunk-5

ggplot(df, aes(x = period, y = value, color = series_code)) +
  geom_line(size = 2) +
  dbnomics()

plot of chunk unnamed-chunk-6

Fetch two series from dataset ‘Unemployment rate’ (ZUTN) of AMECO provider

df <- rdb(ids = c('AMECO/ZUTN/EA19.1.0.0.0.ZUTN', 'AMECO/ZUTN/DNK.1.0.0.0.ZUTN')) %>%
  filter(!is.na(value))

plot of chunk unnamed-chunk-8

ggplot(df, aes(x = period, y = value, color = series_code)) +
  geom_line(size = 2) +
  dbnomics()

plot of chunk unnamed-chunk-9

Fetch two series from different datasets of different providers

df <- rdb(ids = c('AMECO/ZUTN/EA19.1.0.0.0.ZUTN', 'Eurostat/une_rt_q/Q.SA.TOTAL.PC_ACT.T.EA19')) %>%
  filter(!is.na(value))

plot of chunk unnamed-chunk-11

ggplot(df, aes(x = period, y = value, color = series_code)) +
  geom_line(size = 2) +
  dbnomics()

plot of chunk unnamed-chunk-12

Fetch time series by mask

The code mask notation is a very concise way to select one or many time series at once. It is compatible only with some providers : BIS, ECB, Eurostat, FED, ILO, IMF, INSEE, OECD, WTO.

Fetch one series from dataset ‘Consumer Price Index’ (CPI) of IMF

df <- rdb('IMF', 'CPI', mask = 'M.DE.PCPIEC_WT') %>%
  filter(!is.na(value))

plot of chunk unnamed-chunk-14

ggplot(df, aes(x = period, y = value, color = series_code)) +
  geom_step(size = 2) +
  dbnomics()

plot of chunk unnamed-chunk-15

Fetch two series from dataset ‘Consumer Price Index’ (CPI) of IMF

You just have to add a + between two different values of a dimension.

df <- rdb('IMF', 'CPI', mask = 'M.DE+FR.PCPIEC_WT') %>%
  filter(!is.na(value))

plot of chunk unnamed-chunk-17

ggplot(df, aes(x = period, y = value, color = series_code)) +
  geom_step(size = 2) +
  dbnomics()

plot of chunk unnamed-chunk-18

Fetch all series along one dimension from dataset ‘Consumer Price Index’ (CPI) of IMF

df <- rdb('IMF', 'CPI', mask = 'M..PCPIEC_WT') %>%
  filter(!is.na(value)) %>%
  arrange(desc(period), REF_AREA) %>%
  head(100)

plot of chunk unnamed-chunk-20

Fetch series along multiple dimensions from dataset ‘Consumer Price Index’ (CPI) of IMF

df <- rdb('IMF', 'CPI', mask = 'M..PCPIEC_IX+PCPIA_IX') %>%
  filter(!is.na(value)) %>%
  group_by(INDICATOR) %>%
  top_n(n = 50, wt = period)

plot of chunk unnamed-chunk-22

Fetch time series by dimensions

Searching by dimension is a less concise way to select time series than using the code mask, but it works with all the different providers. You have a “Description of series code” at the bottom of each dataset page on the DBnomics website.

Fetch one value of one dimension from dataset ‘Unemployment rate’ (ZUTN) of AMECO provider

df <- rdb('AMECO', 'ZUTN', dimensions = '{"geo": ["ea19"]}') %>%
  filter(!is.na(value))

plot of chunk unnamed-chunk-24

ggplot(df, aes(x = period, y = value, color = series_code)) +
  geom_line(size = 2) +
  dbnomics()

plot of chunk unnamed-chunk-25

Fetch two values of one dimension from dataset ‘Unemployment rate’ (ZUTN) of AMECO provider

df <- rdb('AMECO', 'ZUTN', dimensions = list(geo = c("ea19", "dnk"))) %>%
  filter(!is.na(value))
# or
# df <- rdb('AMECO', 'ZUTN', dimensions = '{"geo": ["ea19", "dnk"]}') %>%
#   filter(!is.na(value))

plot of chunk unnamed-chunk-27

ggplot(df, aes(x = period, y = value, color = series_code)) +
  geom_line(size = 2) +
  dbnomics()

plot of chunk unnamed-chunk-28

Fetch several values of several dimensions from dataset ‘Doing business’ (DB) of World Bank

df <- rdb('WB', 'DB', dimensions = list(country = c("DZ", "PE"), indicator = c("ENF.CONT.COEN.COST.ZS", "IC.REG.COST.PC.FE.ZS"))) %>%
  filter(!is.na(value))
# or
# df <- rdb('WB', 'DB', dimensions = '{"country": ["DZ", "PE"],"indicator": ["ENF.CONT.COEN.COST.ZS","IC.REG.COST.PC.FE.ZS"]}') %>%
#   filter(!is.na(value))

plot of chunk unnamed-chunk-30

ggplot(df, aes(x = period, y = value, color = series_name)) +
  geom_line(size = 2) +
  dbnomics()

plot of chunk unnamed-chunk-31

Fetch time series found on the web site

When you don’t know the codes of the dimensions, provider, dataset or series, you can:

  • go to the page of a dataset on DBnomics website, for example Doing Business,

  • select some dimensions by using the input widgets of the left column,

  • click on “Copy API link” in the menu of the “Download” button,

  • use the rdb_by_api_link function such as below.

df <- rdb_by_api_link("https://api.db.nomics.world/v22/series/WB/DB?dimensions=%7B%22country%22%3A%5B%22FR%22%2C%22IT%22%2C%22ES%22%5D%7D&q=IC.REG.PROC.FE.NO&observations=1&format=json&align_periods=1&offset=0&facets=0") %>%
  filter(!is.na(value))

plot of chunk unnamed-chunk-33

ggplot(df, aes(x = period, y = value, color = series_name)) +
  geom_step(size = 2) +
  dbnomics()

plot of chunk unnamed-chunk-34

Fetch time series from the cart

On the cart page of the DBnomics website, click on “Copy API link” and copy-paste it as an argument of the rdb_by_api_link function. Please note that when you update your cart, you have to copy this link again, because the link itself contains the ids of the series in the cart.



df <- rdb_by_api_link("https://api.db.nomics.world/v22/series?series_ids=BOE%2F8745%2FLPMB23A%2CBOE%2F8745%2FLPMB26A&observations=1&format=json&align_periods=1") %>%
  filter(!is.na(value))
## Error in UseMethod("filter_"): no applicable method for 'filter_' applied to an object of class "NULL"

plot of chunk unnamed-chunk-37

ggplot(df, aes(x = period, y = value, color = series_name)) +
  geom_line(size = 2) +
  scale_y_continuous(labels = function(x) { format(x, big.mark = " ") }) +
  dbnomics()

plot of chunk unnamed-chunk-38

Proxy configuration or connection error Could not resolve host

When using the functions rdb or rdb_..., you may come across the following error :

Error in open.connection(con, "rb") :
  Could not resolve host: api.db.nomics.world

To get round this situation, you have two options :

  1. configure curl to use a specific and authorized proxy.

  2. use the default R internet connection i.e. the Internet Explorer proxy defined in internet2.dll.

Configure curl to use a specific and authorized proxy

In rdbnomics, by default the function curl_fetch_memory (of the package curl) is used to fetch the data. If a specific proxy must be used, it is possible to define it permanently with the package option rdbnomics.curl_config or on the fly through the argument curl_config. In that way the object is passed to the argument handle of the curl_fetch_memory function.
To see the available parameters, run names(curl_options()) in R or visit the website https://curl.haxx.se/libcurl/c/curl_easy_setopt.html. Once they are chosen, you define the curl object as follows :

h <- curl::new_handle(
  proxy = "",
  proxyport = <port>,
  proxyusername = "",
  proxypassword = ""
)

Set the connection up for a session

The curl connection can be set up for a session by modifying the following package option :

options(rdbnomics.curl_config = h)

When fetching the data, the command curl_fetch_memory(url = <...>, handle = h) is executed. In the event that you want to add others arguments, use :

options(rdbnomics.curl_config = list(handle = h, arg = <...>))

After configuration, just use the standard functions of rdbnomics e.g. :

df1 <- rdb(ids = 'AMECO/ZUTN/EA19.1.0.0.0.ZUTN')

This option of the package can be disabled with :

options(rdbnomics.curl = NULL)

Use the connection only for a function call

If a complete configuration is not needed but just an “on the fly” execution, then use the argument curl_config of the functions rdb and rdb_... :

df1 <- rdb(ids = 'AMECO/ZUTN/EA19.1.0.0.0.ZUTN', curl_config = h)

Use the default R internet connection

To retrieve the data with the default R internet connection, rdbnomics will use the base function readLines.

Set the connection up for a session

To activate this feature for a session, you need to enable an option of the package :

options(rdbnomics.use_readLines = TRUE)

And then use the standard function as follows :

df1 <- rdb(ids = 'AMECO/ZUTN/EA19.1.0.0.0.ZUTN')

This configuration can be disabled with :

options(rdbnomics.use_readLines = FALSE)

Use the connection only for a function call

If you just want to do it once, you may use the argument use_readLines of the functions rdb and rdb_... :

df1 <- rdb(ids = 'AMECO/ZUTN/EA19.1.0.0.0.ZUTN', use_readLines = TRUE)

To leave a comment for the author, please follow the link and comment on their blog: Macroeconomic Observatory - R.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.



If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Comments are closed.

Search R-bloggers

Sponsors

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)