Purrring progress bars (adding a progress bar to `purrr::map`)

(This article was first published on R on Adi Sarid's personal blog, and kindly contributed to R-bloggers)

With all the functional programming going on (i.e., purrr::map and the likes), there is at least one thing that I found missing: progress bars. The plyr::do function had a nice looking progress bar open up by default if the operation took more than 2 seconds and had at least two more to go (as per Hadley’s description in Issue#149 in tidyverse/purrr).

The issue is still open, for the time of writing these lines, and will probably be solved sometime in the near future as a feature of purrr::map.

Personally, I like @cderv’s elegent solution suggested at that same github issue.

Here is an example implementation for reading multiple files within a directory and combining them into a single tibble while showing a progress bar when reading the files. The file reading is very similar to what was suggested in this post.

library(purrr)
library(readr)
library(dplyr)

# directory from which to read a bunch of files (the example here uses csv)
file_list <- dir(path = "PATH_TO_DIRECTORY", pattern = ".csv")

# define reading function which includes the progress bar updates and printing
read_with_progress <- function(filename){
  pb$tick()$print()
  data_read <- read_csv(filename)
  # you can add additional operations on data_read, or 
  # decide on entirely different task that this function should do.
}

# create the progress bar with a dplyr function. 
pb <- progress_estimated(length(file_list))
res <- file_list %>%
  map_df(~read_with_progress(.))

That’s it. You’re set to go with a cool progress bar which will print out something like this while the operation is carried out:

|=====================================           |80% ~23 s remaining

To leave a comment for the author, please follow the link and comment on their blog: R on Adi Sarid's personal blog.

R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...



If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Comments are closed.

Search R-bloggers

Sponsors

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)