Windows user space issues with installing R packages

September 17, 2019
By

[This article was first published on Fellgernon Bit - rstats, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Are you a Microsoft Windows R user? Does your Windows username include a space? Like Firstname Lastname. Then you might occassionally run into issues installing packages due to spaces.

Solutions

You could either re-install Windows with a username that has no spaces such as Lastname 1, but that’s probably not an easy option. Or you can:

  • Edit your TMP and TEMP environment variables to a location with no spaces, like C:\TEMP following instructions like these ones.
  • Preferably install R at a location with no spaces, like C:\R, instead of the default C:\Program Files 2.

Backstory

A co-worker wanted to install the clusterprofiler Bioconductor package which depends on the DO.db Bioconductor package. This co-worker uses a Windows machine that has a username with a space. Let’s say it was me with Leo Collado to keep them anonymous. The DO.db is only available as a “Source” package with no Windows binary as you can see here.

This means that R has to:

  1. download a tar.gz file,
  2. uncompress it,
  3. and then install it.

In particular, we are talking about DO.db_2.9.tar.gz in this case.

The installation instructions for DO.db are:

if (!requireNamespace("BiocManager", quietly = TRUE))
    install.packages("BiocManager")

BiocManager::install("DO.db")

Uncompressing

Internally, BiocManager::install() ends up using utils::install.packages(). The first step, downloading, works well. Uncompressing a file in this scenario fails. Why?

> BiocManager::install('DO.db', lib = 'C:/R/R-3.6.0/library')
Bioconductor version 3.9 (BiocManager 1.30.4), R 3.6.0 (2019-04-26)
Installing package(s) 'BiocVersion', 'DO.db'

## removed output

trying URL 'https://bioconductor.org/packages/3.9/data/annotation/src/contrib/DO.db_2.9.tar.gz'
Content type 'application/x-gzip' length 1769978 bytes (1.7 MB)
downloaded 1.7 MB

Error in untar2(tarfile, files, list, exdir, restore_times) :
  incomplete block on file

The downloaded source packages are in
        ‘C:\Users\Leo Collado\AppData\Local\Temp\RtmpqiBJ53\downloaded_packages’

If you search on Google the error message you’ll find links like this one which hint towards an incomplete download. But the download works. You can even download the file and try to run untar() manually and it will fail.

We were told to try installing R at a location with no spaces, so by this point, R was installed at C:\R\R-3.6.0\, hence the lib specification you see above, though it’s irrelevant for these errors.

Uncompressing the tar.gz file is done by utils::untar(). If you look at the code for utils::untar() you’ll see:

## The function definition of utils::untar
function (tarfile, files = NULL, list = FALSE, exdir = ".", compressed = NA, 
  extras = NULL, verbose = FALSE, restore_times = TRUE,
  support_old_tars = Sys.getenv("R_SUPPORT_OLD_TARS", 
    FALSE), tar = Sys.getenv("TAR")) 

## Inside utils::untar()
if (inherits(tarfile, "connection") || identical(tar, "internal")) {
  if (!missing(compressed)) 
    warning("argument 'compressed' is ignored for the internal method")
  return(untar2(tarfile, files, list, exdir, restore_times))
}

## Further below
TAR <- tar
if (!nzchar(TAR) && .Platform$OS.type == "windows" && nzchar(Sys.which("tar.exe"))) 
  TAR <- "tar.exe"
if (!nzchar(TAR) || TAR == "internal") 
  return(untar2(tarfile, files, list, exdir))

In this case, the first untar2() call is called. That is: return(untar2(tarfile, files, list, exdir, restore_times)). The error message incomplete block on file is not really informative in this case because untar2() is not happy when there’s a space in the path to the file 3.

We can get around this untar2() issue by uncompressing the tar.gz file ourselves in a path that has no spaces. For example, if we download DO.db_2.9.tar.gz to C:\R we can uncompress the tar.gz file with:

utils::untar('C:/R/DO.db_2.9.tar.gz')

Installation

Let’s proceed to installing the package.

> install.packages('C:/R/DO.db', repos = NULL, type = 'source', lib = 'C:/R/R-3.6.0/library')
* installing *source* package 'DO.db' ...
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading
ARGUMENT 'Collado\AppData\Local\Temp\Rtmp8EQDjB\Rin2ef05088650f' __ignored__

Error: object 'ÿþ' not found
Execution halted

ERROR: lazy loading failed for package 'DO.db'
* removing 'C:/R/R-3.6.0/library/DO.db'

Warning message:
In install.packages("C:/R/DO.db", repos = NULL, type = "source",  :
  installation of package ‘C:/R/DO.db’ had non-zero exit status
>

Oh noes! It didn’t work 😖 What happened?

If you look closely, you’ll see that it says ARGUMENT 'Collado\AppData\Local\Temp\Rtmp8EQDjB\Rin2ef05088650f' __ignored__. Wait! Collado is the Lastname portion of the username! So we have another space issue 4. That structure though looks very familiar, it’s from base::tempdir()!

> tempdir()

[1] "C:\\Users\\Leo Collado\\AppData\\Local\\Temp\\RtmpqiBJ53"

The help file for ?tempdir contained the clues to solving this issue.

By default, tmpdir will be the directory given by tempdir().
This will be a subdirectory of the per-session temporary directory
found by the following rule when the R session is started.
The environment variables TMPDIR, TMP and TEMP are checked in turn
and the first found which points to a writable directory is used:
if none succeeds ‘/tmp’ is used. The path should not contain spaces.
Note that setting any of these environment variables in the R session
has no effect on tempdir(): the per-session temporary directory is
created before the interpreter is started.

We can set the TMPDIR environment variable which will be used by the R session spawned inside the installation of DO.db and… it works!

> Sys.setenv(TMPDIR = 'C:/R/tmp_leo')
> Sys.getenv('TMPDIR')
[1] "C:/R/tmp_leo"
>
>
> install.packages('C:/R/DO.db', repos = NULL, type = 'source', lib = 'C:/R/R-3.6.0/library')
* installing *source* package 'DO.db' ...
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
  converting help for package 'DO.db'
    finding HTML links ... done
    DOANCESTOR                              html 
    DOBASE                                  html 
    DOCHILDREN                              html 
    DOMAPCOUNTS                             html 
    DOOBSOLETE                              html 
    DOOFFSPRING                             html 
    DOPARENTS                               html 
    DOSYNONYM                               html  
    DOTERM                                  html 
    DOTerms-class                           html 
    DOTermsAnnDbBimap                       html 
    DO_dbconn                               html 
** building package indices
** testing if installed package can be loaded from temporary location
*** arch - i386
*** arch - x64
** testing if installed package can be loaded from final location
*** arch - i386
*** arch - x64
** testing if installed package keeps a record of temporary installation path
* DONE (DO.db)
Making 'packages.html' ... done

clusterProfiler installation

Now we can continue and install clusterProfiler, right?

> BiocManager::install('clusterProfiler', lib = 'C:/R/R-3.6.0/library')
Bioconductor version 3.9 (BiocManager 1.30.4), R 3.6.0 (2019-04-26)
Installing package(s) 'clusterProfiler'
also installing the dependencies ‘sys’, ‘formatR’, ‘askpass’, ‘farver’, ‘backports’, ‘zeallot’, ‘lambda.r’, ‘futile.options’, ‘curl’, ‘mime’, ‘openssl’, ‘hms’, ‘triebeard’, ‘tweenr’, ‘polyclip’, ‘RcppEigen’, ‘colorspace’, ‘utf8’, ‘vctrs’, ‘futile.logger’, ‘snow’, ‘data.table’, ‘fastmatch’, ‘stringr’, ‘httr’, ‘jsonlite’, ‘progress’, ‘urltools’, ‘xml2’, ‘gridGraphics’, ‘ggforce’, ‘ggrepel’, ‘viridis’, ‘labeling’, ‘munsell’, ‘R6’, ‘cli’, ‘crayon’, ‘fansi’, ‘pillar’, ‘BiocParallel’, ‘fgsea’, ‘reshape2’, ‘cowplot’, ‘europepmc’, ‘ggplotify’, ‘ggraph’, ‘ggridges’, ‘gridExtra’, ‘igraph’, ‘purrr’, ‘RColorBrewer’, ‘UpSetR’, ‘gtable’, ‘lazyeval’, ‘rlang’, ‘scales’, ‘tibble’, ‘viridisLite’, ‘withr’, ‘dplyr’, ‘glue’, ‘stringi’, ‘tidyselect’, ‘DOSE’, ‘enrichplot’, ‘ggplot2’, ‘GO.db’, ‘GOSemSim’, ‘plyr’, ‘qvalue’, ‘rvcheck’, ‘tidyr’

## Delete more output

The downloaded binary packages are in
        C:\Users\Leo Collado\AppData\Local\Temp\RtmpqiBJ53\downloaded_packages
installing the source packages ‘pillar’, ‘GO.db’


trying URL 'https://cloud.r-project.org/src/contrib/pillar_1.4.1.tar.gz'
Content type 'application/x-gzip' length 228572 bytes (223 KB)
downloaded 223 KB

trying URL 'https://bioconductor.org/packages/3.9/data/annotation/src/contrib/GO.db_3.8.2.tar.gz'
Content type 'application/x-gzip' length 31820866 bytes (30.3 MB)
downloaded 30.3 MB

Error in untar2(tarfile, files, list, exdir, restore_times) :
  incomplete block on file

Error in untar2(tarfile, files, list, exdir, restore_times) :
  incomplete block on file

The downloaded source packages are in
        ‘C:\Users\Leo Collado\AppData\Local\Temp\RtmpqiBJ53\downloaded_packages’

The issue is again that utils:::untar2() and thus utils::untar() does not like spaces in the paths. If we look at where the packages were downloaded more closely, we can see a space there at C:\Users\Leo Collado\AppData\Local\Temp\RtmpqiBJ53\downloaded_packages. If you check the help file for utils::install.packages() you’ll see that destdir controls this:

destdir

directory where downloaded packages are stored. If it is NULL
(the default) a subdirectory downloaded_packages of the session
temporary directory will be used (and the files will be deleted
at the end of the session).

If we dig into utils::install.packages() we can see how this comes to play.

## Part of utils::install.packages()
if (is.null(destdir) && nonlocalrepos) {
  tmpd <- file.path(tempdir(), "downloaded_packages")
  if (!file.exists(tmpd) && !dir.create(tmpd)) 
    stop(gettextf("unable to create temporary directory %s", 
      sQuote(tmpd)), domain = NA)
}

Setting the environment variable TMPDIR doesn’t work here as the instructions for tempdir() specify 5 although I now see that you can edit the .Renviron file as instructed here.

In any case, if we specify a destdir without spaces we overide the need to control tempdir(), enable utils::untar() to work and we can finally install clusterProfiler 🎉.

> BiocManager::install('clusterProfiler', lib = 'C:/R/R-3.6.0/library', destdir = 'C:/R/dest_leo')
Bioconductor version 3.9 (BiocManager 1.30.4), R 3.6.0 (2019-04-26)
Installing package(s) 'clusterProfiler'
also installing the dependency ‘GO.db’

trying URL 'https://bioconductor.org/packages/3.9/bioc/bin/windows/contrib/3.6/clusterProfiler_3.12.0.zip'
Content type 'application/zip' length 623524 bytes (608 KB)
downloaded 608 KB

package ‘clusterProfiler’ successfully unpacked and MD5 sums checked
installing the source package ‘GO.db’

trying URL 'https://bioconductor.org/packages/3.9/data/annotation/src/contrib/GO.db_3.8.2.tar.gz'
Content type 'application/x-gzip' length 31820866 bytes (30.3 MB)
downloaded 30.3 MB

* installing *source* package 'GO.db' ...
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
  converting help for package 'GO.db'
    finding HTML links ... done
    GOBASE                                  html 
    GOBPANCESTOR                            html 
    GOBPCHILDREN                            html 
    GOBPOFFSPRING                           html 
    GOBPPARENTS                             html 
    GOCCANCESTOR                            html 
    GOCCCHILDREN                            html 
    GOCCOFFSPRING                           html 
    GOCCPARENTS                             html 
    GOMAPCOUNTS                             html 
    GOMFANCESTOR                            html 
    GOMFCHILDREN                            html 
    GOMFOFFSPRING                           html 
    GOMFPARENTS                             html 
    GOOBSOLETE                              html 
    GOSYNONYM                               html 
    GOTERM                                  html  
    GO_dbconn                               html 
** building package indices
** testing if installed package can be loaded from temporary location
*** arch - i386
*** arch - x64
** testing if installed package can be loaded from final location
*** arch - i386
*** arch - x64
** testing if installed package keeps a record of temporary installation path
* DONE (GO.db)
Making 'packages.html' ... done

Closing

All of the above seemed like too much. In addition, it seemed like BiocManager::install('hypeR', destdir = 'C:/R/dest_leo') was not working 6. I likely missed something here earlier today. So controlling utils::tempdir() seemed like the easiest solution such that the defaults of where a package gets downloaded, uncompressed, etc all worked. And the simplest solution we thought of was to create the C:\TEMP directory and update the Windows environment variables TMP and TEMP to point to that location. Then, the rest of the commands worked without having to specify lib or destdir or manually run utils::untar().

As a whole, remember to look for spaces in the error messages! This is specially relevant when you are having issues as a Microsoft Windows R user.

If you have other solutions for Microsoft Windows R users with usernames that have at least one space, please let us know in the comments! Thank you! 🙌🏽

References

[1]
C. Boettiger.
knitcitations: Citations for ‘Knitr’ Markdown Files.
R package version 1.0.8.
2017.
URL: https://CRAN.R-project.org/package=knitcitations.

[2]
G. Csárdi, R. core, H. Wickham, W. Chang, et al.
sessioninfo: R Session Information.
R package version 1.1.1.
2018.
URL: https://CRAN.R-project.org/package=sessioninfo.

[3]
A. Oleś, M. Morgan, and W. Huber.
BiocStyle: Standard styles for vignettes and other Bioconductor documents.
R package version 2.12.0.
2019.
URL: https://github.com/Bioconductor/BiocStyle.

[4]
Y. Xie, A. P. Hill, and A. Thomas.
blogdown: Creating Websites with R Markdown.
ISBN 978-0815363729.
Boca Raton, Florida: Chapman and Hall/CRC, 2017.
URL: https://github.com/rstudio/blogdown.

Reproducibility

## ─ Session info ───────────────────────────────────────────────────────────────────────────────────────────────────────
##  setting  value                       
##  version  R version 3.6.1 (2019-07-05)
##  os       macOS Mojave 10.14.6        
##  system   x86_64, darwin15.6.0        
##  ui       X11                         
##  language (EN)                        
##  collate  en_US.UTF-8                 
##  ctype    en_US.UTF-8                 
##  tz       America/New_York            
##  date     2019-09-18                  
## 
## ─ Packages ───────────────────────────────────────────────────────────────────────────────────────────────────────────
##  package       * version date       lib source                            
##  assertthat      0.2.1   2019-03-21 [1] CRAN (R 3.6.0)                    
##  bibtex          0.4.2   2017-06-30 [1] CRAN (R 3.6.0)                    
##  BiocManager     1.30.4  2018-11-13 [1] CRAN (R 3.6.0)                    
##  BiocStyle     * 2.12.0  2019-05-02 [1] Bioconductor                      
##  blogdown        0.15    2019-08-21 [1] CRAN (R 3.6.1)                    
##  bookdown        0.13    2019-08-21 [1] CRAN (R 3.6.0)                    
##  cli             1.1.0   2019-03-19 [1] CRAN (R 3.6.0)                    
##  colorout      * 1.2-1   2019-05-07 [1] Github (jalvesaq/[email protected])
##  crayon          1.3.4   2017-09-16 [1] CRAN (R 3.6.0)                    
##  digest          0.6.20  2019-07-04 [1] CRAN (R 3.6.0)                    
##  evaluate        0.14    2019-05-28 [1] CRAN (R 3.6.0)                    
##  htmltools       0.3.6   2017-04-28 [1] CRAN (R 3.6.0)                    
##  httr            1.4.1   2019-08-05 [1] CRAN (R 3.6.0)                    
##  jsonlite        1.6     2018-12-07 [1] CRAN (R 3.6.0)                    
##  knitcitations * 1.0.8   2017-07-04 [1] CRAN (R 3.6.0)                    
##  knitr           1.24    2019-08-08 [1] CRAN (R 3.6.0)                    
##  lubridate       1.7.4   2018-04-11 [1] CRAN (R 3.6.0)                    
##  magrittr        1.5     2014-11-22 [1] CRAN (R 3.6.0)                    
##  plyr            1.8.4   2016-06-08 [1] CRAN (R 3.6.0)                    
##  R6              2.4.0   2019-02-14 [1] CRAN (R 3.6.0)                    
##  Rcpp            1.0.2   2019-07-25 [1] CRAN (R 3.6.0)                    
##  RefManageR      1.2.12  2019-04-03 [1] CRAN (R 3.6.0)                    
##  rmarkdown       1.15    2019-08-21 [1] CRAN (R 3.6.1)                    
##  sessioninfo   * 1.1.1   2018-11-05 [1] CRAN (R 3.6.0)                    
##  stringi         1.4.3   2019-03-12 [1] CRAN (R 3.6.0)                    
##  stringr         1.4.0   2019-02-10 [1] CRAN (R 3.6.0)                    
##  withr           2.1.2   2018-03-15 [1] CRAN (R 3.6.0)                    
##  xfun            0.9     2019-08-21 [1] CRAN (R 3.6.0)                    
##  xml2            1.2.2   2019-08-09 [1] CRAN (R 3.6.0)                    
##  yaml            2.2.0   2018-07-25 [1] CRAN (R 3.6.0)                    
## 
## [1] /Library/Frameworks/R.framework/Versions/3.6/Resources/library

  1. This is the case in the Bioconductor Windows build machine where the username is biocbuild as you can see here.

  2. In the Bioconductor Windows build machines there are again no spaces in the path to the R installation and the library where packages are installed.

  3. Hopefully in the future Google will lead you to this blog post and you might avoid the rabbit hole I went through!

  4. By the way, at this point I thought that the error was related to Error: object 'ÿþ' not found and maybe some encoding issues since the DO.db package has Chinese characters.

  5. Didn’t stop me from trying hehe. I tried using usethis::edit_r_profile() and adding Sys.setenv(TMPDIR = 'C:/R/tmp_leo') but that didn’t work.

  6. I would need to test this more before reporting it properly to Bioconductor.

To leave a comment for the author, please follow the link and comment on their blog: Fellgernon Bit - rstats.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.



If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Comments are closed.

Search R-bloggers

Sponsors

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)