Introducing the ‘gimms’ package

[This article was first published on metvurst, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

This is a guest post by Florian Detsch

What it is all about

With the most recent update of the AVHRR GIMMS data collection to NDVI3g (Pinzon and Tucker, 2014), we decided to create a package from all functions we have written so far to download and process GIMMS binary files from the NASA ECOCAST server. The package is called gimms and features a collection of fundamental work steps required to get the data into R:

  • updateInventory to list all GIMMS files available online and
  • rearrangeFiles to sort (online or local) files by date,
  • downloadGimms to download selected files,
  • rasterizeGimms to import the binary data as 'Raster*' objects into R and
  • monthlyComposite to aggregate the bi-monthly datasets to monthly value
    composites.

How to install

The gimms package (version 0.1.1) is now officially on CRAN and can be installed directly via

## install 'gimms' package
install.packages("gimms")

## load 'gimms' package
library(gimms)

In order to use the development version (no liability assumed), please refer to the 'develop' branch hosted at GitHub. There, you will also find the latest news and updates concerning the package.

library(devtools)
install_github("environmentalinformatics-marburg/gimms", ref = "develop")

List available files

updateInventory imports the latest version of the online file inventory as 'character' vector into R. By setting sort = TRUE, it is at the same time a wrapper around rearrangeFiles as the output vector will be sorted by date rather than in alphabetical order. The latter feature proves particularly useful when considering the GIMMS file naming convention, where e.g. 'geo13jul15a.n19-VI3g' means the first half of July 2013. In case no active internet connection is available, updateInventory automatically imports the latest offline version of the file inventory.

gimms_files <- updateInventory(sort = TRUE)

## Trying to update GIMMS inventory from server...
## Online update of the GIMMS file inventory successful!

gimms_files[1:5]

## [1] "http://ecocast.arc.nasa.gov/data/pub/gimms/3g.v0/1980s_new/geo81jul15a.n07-VI3g"
## [2] "http://ecocast.arc.nasa.gov/data/pub/gimms/3g.v0/1980s_new/geo81jul15b.n07-VI3g"
## [3] "http://ecocast.arc.nasa.gov/data/pub/gimms/3g.v0/1980s_new/geo81aug15a.n07-VI3g"
## [4] "http://ecocast.arc.nasa.gov/data/pub/gimms/3g.v0/1980s_new/geo81aug15b.n07-VI3g"
## [5] "http://ecocast.arc.nasa.gov/data/pub/gimms/3g.v0/1980s_new/geo81sep15a.n07-VI3g"

Download files

The next logical step of the gimms processing chain is to download selected (if not all) bi-monthly datasets. This can be achieved by running downloadGimms which accepts various types of input parameters.

  • 'missing' input → download entire collection
    Specifying no particular input is possibly the most straightforward way of data acquisition. The function will automatically start to download the entire collection of files (currently July 1981 to December 2013) and write the data to dsn.
## download entire gimms collection
downloadGimms(dsn = paste0(getwd(), "/data"))
  • 'numeric' input → download temporal range
    It is also possibly to specify a start year (x) and/or end year (y) to limit the temporal coverage of the datasets to be downloaded. In case x (or y) is missing, data download will automatically start from the first (or finish with the last) year available.
## download gimms data from 1998-2000
downloadGimms(x = 1998, y = 2000, 
              dsn = paste0(getwd(), "/data"))
  • 'character' input → download particular files
    As a third and final possibility to run downloadGimms, it is also possible to supply a 'character' vector consisting of valid online filepaths. The latter can easily be retrieved from updateInventory (as demonstrated above) and directly passed on to the input argument x.
## download manually selected files
downloadGimms(x = gimms_files[769:780], 
              dsn = paste0(getwd(), "/data"))

Rasterize downloaded data

rasterizeGimms transforms the retrieved GIMMS data from native binary into common 'GeoTiff' format and makes the single layers available in R as ordinary 'Raster*' objects. Thereby, it is up to the user to decide whether or not to discard 'mask-water' values (-10,000) and 'mask-nodata' values (-5,000) (see also the official NDVI3g README) and apply the scaling factor (1/10,000). Since rasterizing usually takes some time, we highly recommend to make use of the filename argument that automatically invokes raster::writeRaster.

## list available files
gimms_files <- rearrangeFiles(dsn = paste0(getwd(), "/data"), 
                              pattern = "^geo13", full.names = TRUE)

## rasterize files
gimms_raster <- rasterizeGimms(gimms_files, 
                               filename = paste0(gimms_files, ".tif"))

With a little bit of effort and the help of RColorBrewer and sp, here is what we have created so far.

spplot

Figure 1.Global bi-monthly GIMMS NDVI3g images from July to December 2013.

Generate monthly composites

Sometimes, the user is required to calculate monthly value composites from the bi-monthly GIMMS datasets, e.g. to ensure temporal overlap with some other remote sensing product. For that purpose, gimms also features a function called monthlyComposite which works both on vectors of filenames and entire 'RasterStack' objects (ideally returned by rasterizeGimms) and calculates monthly values based on a user-defined function (e.g. fun = max to create monthly MVC layers). The function is heavily based on stackApply from the raster package and the required coding work is quite straightforward.

## 'GeoTiff' files created during the previous step
gimms_files_tif <- sapply(gimms_raster@layers, function(i) attr(i@file, "name"))

## create monthly maximum value composites
gimms_raster_mvc <- monthlyComposite(gimms_files_tif)

spplot

Figure 2.Global monthly composite GIMMS NDVI3g images from July to December 2013.

Final remark

A more comprehensive version of this short introduction to the gimms package including a collection of use cases (particularly in conjunction with R's parallel capabilities) can be found online at https://github.com/environmentalinformatics-marburg/gimms. Any comments on how to improve the package, possible bug-reports etc. are highly appreciated!

To leave a comment for the author, please follow the link and comment on their blog: metvurst.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)