Installing Package Dependencies without external http(s) requests
[This article was first published on Peter E. DeWitt, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.
Consider you have a server that is running behind a firewall and, for security
reasons, cannot make external http(s) requests. Further, you have R running on
this server and you need to install a set of packages. The simple approach of
install.packages("<pkg-name>", repo = "<favorite cran mirror>")is not an option since you will have no access to the CRAN repository.
Another option would be to download the source files (.tar.gz files) form CRAN
or BioConductor, transfer those files to the sever via FTP, and then install the
packages via
install.packages("<path to pkg-name_version.tar.gz", type = "source")This approach will work well, with one big exception, the dependencies of the
package may not be on the server. How do you get the source files for all the
dependencies of your package? What about the dependencies of the dependencies,
and the dependencies of the dependencies of the dependencies? Simply, how do
you install R packages on a machine that is not allowed to make external http(s)
requests?
Here is how I approached this problem. On my local machine, a machine with
internet access, I ran a script (a script that will be shown and explained in
detail below) which will download all the dependencies and dependencies of
dependencies, etc., from both CRAN and BioConductor, and generate a makefile
to install the packages in the correct order, i.e., in an order such that the
dependencies are met.
When the script finished, the source files and the makefile can be transfered
to the server without external http(s) request authority. Running the
makefile will install the packages, and is an easy way to track and report
install errors.
We need to define what constitutes a dependency. In a package DESCRIPTION
file packages listed under the field Depends, Imports, and LinkingTo are
what we will consider dependencies. Suggests and Enhances are omitted as
they are not needed for the package to work.
Build Dependencies
An R script build-dep-list.R has been written and is expected to be evaluated
from the command line via
Rscript --vanilla build-dep-list.R [pkg1] [pkg2] [...] [pkgn]Where pkg1 is the name of the first known package to download, pkg2 the
second known package to download, …, and pkgn the nth package to download.
The script will download all the dependencies for pkg1, ..., pkgn, and the
dependencies of the dependencies, and so on. The script will also generate a
makefile to help with the installation of the packages, aiming to get the
order of the installs so that the install of pkg1, ..., pkgn will not
error.
The full script can be found on my
github page. The script
will be broken up into pieces here with additional detail and explanation.
When I develop scripts that I expect to evaluated in the terminal, I will start
the script with a check of interactive(). If in an interactive session we’ll
have set variables to values needed for testing and development, and if not in
an interactive session we’ll use the command line arguments to define the value
of the variables. This could also be edited so that the expected evaluation
would be done in an interactive session. For then work we will have the
character vector OUR_PACKAGES to store the names of the packages we
want/need to install.
<span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="nf">interactive</span><span class="p">())</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="n">OUR_PACKAGES</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="nf">c</span><span class="p">(</span><span class="s2">"graph"</span><span class="p">,</span><span class="w"> </span><span class="s2">"gRbase"</span><span class="p">,</span><span class="w"> </span><span class="s2">"gRain"</span><span class="p">,</span><span class="w"> </span><span class="s2">"jsonlite"</span><span class="p">,</span><span class="w"> </span><span class="s2">"plotly"</span><span class="p">,</span><span class="w"> </span><span class="s2">"SHELF"</span><span class="p">,</span><span class="w">
</span><span class="s2">"rjson"</span><span class="p">,</span><span class="w"> </span><span class="s2">"svglite"</span><span class="p">,</span><span class="w"> </span><span class="s2">"magrittr"</span><span class="p">)</span><span class="w">
</span><span class="p">}</span><span class="w"> </span><span class="k">else</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="n">OUR_PACKAGES</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">commandArgs</span><span class="p">(</span><span class="n">trailingOnly</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">TRUE</span><span class="p">)</span><span class="w">
</span><span class="p">}</span><span class="w">
</span>We also need to define the repositories which we will query for the packages.
We’ll use RStudio’s CRAN mirror and the repository for BioConductor.
<span class="c1"># Repositories to look for packages</span><span class="w">
</span><span class="n">CRAN</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="s2">"https://cran.rstudio.com/"</span><span class="w">
</span><span class="n">BIOC</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="s2">"https://bioconductor.org/packages/release/bioc/"</span><span class="w">
</span>Now, let’s look into the packages. Packages are classified into three priority
classes, “base”, “recommended”, and “NA”. The “base” packages are standard an R
installation, and the ‘recommended’ are in any standard installation of R. All
other packages have Priority == NA.
<span class="n">ipkgs</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">utils</span><span class="o">::</span><span class="n">installed.packages</span><span class="p">()</span><span class="w">
</span><span class="n">ipkgs</span><span class="p">[</span><span class="n">ipkgs</span><span class="p">[,</span><span class="w"> </span><span class="s2">"Priority"</span><span class="p">]</span><span class="w"> </span><span class="o">%in%</span><span class="w"> </span><span class="s2">"base"</span><span class="p">,</span><span class="w"> </span><span class="s2">"Package"</span><span class="p">]</span><span class="w">
</span><span class="c1">## base compiler datasets graphics grDevices grid </span><span class="w">
</span><span class="c1">## "base" "compiler" "datasets" "graphics" "grDevices" "grid" </span><span class="w">
</span><span class="c1">## methods parallel splines stats stats4 tcltk </span><span class="w">
</span><span class="c1">## "methods" "parallel" "splines" "stats" "stats4" "tcltk" </span><span class="w">
</span><span class="c1">## tools utils </span><span class="w">
</span><span class="c1">## "tools" "utils"</span><span class="w">
</span><span class="n">ipkgs</span><span class="p">[</span><span class="n">ipkgs</span><span class="p">[,</span><span class="w"> </span><span class="s2">"Priority"</span><span class="p">]</span><span class="w"> </span><span class="o">%in%</span><span class="w"> </span><span class="s2">"recommended"</span><span class="p">,</span><span class="w"> </span><span class="s2">"Package"</span><span class="p">]</span><span class="w">
</span><span class="c1">## boot class cluster codetools foreign </span><span class="w">
</span><span class="c1">## "boot" "class" "cluster" "codetools" "foreign" </span><span class="w">
</span><span class="c1">## KernSmooth lattice MASS Matrix mgcv </span><span class="w">
</span><span class="c1">## "KernSmooth" "lattice" "MASS" "Matrix" "mgcv" </span><span class="w">
</span><span class="c1">## nnet rpart spatial survival </span><span class="w">
</span><span class="c1">## "nnet" "rpart" "spatial" "survival"</span><span class="w">
</span>Some packages will have dependencies on the “base” and/or “recommended”
packages. We will need to know these packages and omit them form the packages
we will need to download and install.
<span class="n">base_pkgs</span><span class="w"> </span><span class="o"><-</span><span class="w">
</span><span class="n">unname</span><span class="p">(</span><span class="n">utils</span><span class="o">::</span><span class="n">installed.packages</span><span class="p">()[</span><span class="n">utils</span><span class="o">::</span><span class="n">installed.packages</span><span class="p">()[,</span><span class="w"> </span><span class="s2">"Priority"</span><span class="p">]</span><span class="w"> </span><span class="o">%in%</span><span class="w"> </span><span class="nf">c</span><span class="p">(</span><span class="s2">"base"</span><span class="p">,</span><span class="w"> </span><span class="s2">"recommended"</span><span class="p">),</span><span class="w"> </span><span class="s2">"Package"</span><span class="p">])</span><span class="w">
</span>Next step, get a list of the available packages from CRAN and BioConductor. The
return from available.packages is a matrix with all the information we will
need about the packages.
<span class="n">available_pkgs</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">available.packages</span><span class="p">(</span><span class="n">repos</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">c</span><span class="p">(</span><span class="n">CRAN</span><span class="p">,</span><span class="w"> </span><span class="n">BIOC</span><span class="p">))</span><span class="w">
</span><span class="n">str</span><span class="p">(</span><span class="n">available_pkgs</span><span class="p">)</span><span class="w">
</span><span class="c1">## chr [1:13659, 1:17] "A3" "abbyyR" "abc" "abc.data" "ABC.RAP" ...</span><span class="w">
</span><span class="c1">## - attr(*, "dimnames")=List of 2</span><span class="w">
</span><span class="c1">## ..$ : chr [1:13659] "A3" "abbyyR" "abc" "abc.data" ...</span><span class="w">
</span><span class="c1">## ..$ : chr [1:17] "Package" "Version" "Priority" "Depends" ...</span><span class="w">
</span><span class="n">available_pkgs</span><span class="p">[</span><span class="n">available_pkgs</span><span class="p">[,</span><span class="w"> </span><span class="s2">"Package"</span><span class="p">]</span><span class="w"> </span><span class="o">%in%</span><span class="w"> </span><span class="n">OUR_PACKAGES</span><span class="p">,</span><span class="w">
</span><span class="nf">c</span><span class="p">(</span><span class="s2">"Package"</span><span class="p">,</span><span class="w"> </span><span class="s2">"Version"</span><span class="p">,</span><span class="w"> </span><span class="s2">"Depends"</span><span class="p">,</span><span class="w"> </span><span class="s2">"Imports"</span><span class="p">,</span><span class="w"> </span><span class="s2">"LinkingTo"</span><span class="p">,</span><span class="w">
</span><span class="s2">"Repository"</span><span class="p">)]</span><span class="w">
</span><span class="c1">## Package Version </span><span class="w">
</span><span class="c1">## gRain "gRain" "1.3-0" </span><span class="w">
</span><span class="c1">## gRbase "gRbase" "1.8-3" </span><span class="w">
</span><span class="c1">## jsonlite "jsonlite" "1.5" </span><span class="w">
</span><span class="c1">## magrittr "magrittr" "1.5" </span><span class="w">
</span><span class="c1">## plotly "plotly" "4.7.1" </span><span class="w">
</span><span class="c1">## rjson "rjson" "0.2.15"</span><span class="w">
</span><span class="c1">## SHELF "SHELF" "1.3.0" </span><span class="w">
</span><span class="c1">## svglite "svglite" "1.2.1" </span><span class="w">
</span><span class="c1">## graph "graph" "1.56.0"</span><span class="w">
</span><span class="c1">## Depends </span><span class="w">
</span><span class="c1">## gRain "R (>= 3.0.2), methods, gRbase (>= 1.7-2)" </span><span class="w">
</span><span class="c1">## gRbase "R (>= 3.0.2), methods" </span><span class="w">
</span><span class="c1">## jsonlite "methods" </span><span class="w">
</span><span class="c1">## magrittr NA </span><span class="w">
</span><span class="c1">## plotly "R (>= 3.2.0), ggplot2 (>= 2.2.1)" </span><span class="w">
</span><span class="c1">## rjson "R (>= 3.1.0)" </span><span class="w">
</span><span class="c1">## SHELF "R (>= 3.3.1)" </span><span class="w">
</span><span class="c1">## svglite "R (>= 3.0.0)" </span><span class="w">
</span><span class="c1">## graph "R (>= 2.10), methods, BiocGenerics (>= 0.13.11)"</span><span class="w">
</span><span class="c1">## Imports </span><span class="w">
</span><span class="c1">## gRain "igraph, graph, magrittr, functional, Rcpp (>= 0.11.1)" </span><span class="w">
</span><span class="c1">## gRbase "graph, igraph, magrittr, Matrix, RBGL, Rcpp (>= 0.11.1)" </span><span class="w">
</span><span class="c1">## jsonlite NA </span><span class="w">
</span><span class="c1">## magrittr NA </span><span class="w">
</span><span class="c1">## plotly "tools, scales, httr, jsonlite, magrittr, digest, viridisLite,\nbase64enc, htmltools, htmlwidgets (>= 0.9), tidyr, hexbin,\nRColorBrewer, dplyr, tibble, lazyeval (>= 0.2.0), crosstalk,\npurrr, data.table"</span><span class="w">
</span><span class="c1">## rjson NA </span><span class="w">
</span><span class="c1">## SHELF "ggplot2, grid, shiny, stats, graphics, tidyr, MASS, ggExtra" </span><span class="w">
</span><span class="c1">## svglite "Rcpp, gdtools (>= 0.1.6)" </span><span class="w">
</span><span class="c1">## graph "stats, stats4, utils" </span><span class="w">
</span><span class="c1">## LinkingTo </span><span class="w">
</span><span class="c1">## gRain "Rcpp (>= 0.11.1), RcppArmadillo, RcppEigen, gRbase (>=\n1.8-0)"</span><span class="w">
</span><span class="c1">## gRbase "Rcpp (>= 0.11.1), RcppArmadillo, RcppEigen" </span><span class="w">
</span><span class="c1">## jsonlite NA </span><span class="w">
</span><span class="c1">## magrittr NA </span><span class="w">
</span><span class="c1">## plotly NA </span><span class="w">
</span><span class="c1">## rjson NA </span><span class="w">
</span><span class="c1">## SHELF NA </span><span class="w">
</span><span class="c1">## svglite "Rcpp, gdtools, BH" </span><span class="w">
</span><span class="c1">## graph NA </span><span class="w">
</span><span class="c1">## Repository </span><span class="w">
</span><span class="c1">## gRain "https://cran.rstudio.com/src/contrib" </span><span class="w">
</span><span class="c1">## gRbase "https://cran.rstudio.com/src/contrib" </span><span class="w">
</span><span class="c1">## jsonlite "https://cran.rstudio.com/src/contrib" </span><span class="w">
</span><span class="c1">## magrittr "https://cran.rstudio.com/src/contrib" </span><span class="w">
</span><span class="c1">## plotly "https://cran.rstudio.com/src/contrib" </span><span class="w">
</span><span class="c1">## rjson "https://cran.rstudio.com/src/contrib" </span><span class="w">
</span><span class="c1">## SHELF "https://cran.rstudio.com/src/contrib" </span><span class="w">
</span><span class="c1">## svglite "https://cran.rstudio.com/src/contrib" </span><span class="w">
</span><span class="c1">## graph "https://bioconductor.org/packages/release/bioc/src/contrib"</span><span class="w">
</span>In this example we see that the packages listed in OUR_PACKAGES except the
graph package can be downloaded from CRAN. graph and at least one
dependencies, BiocGenerics will need to be downloaded from BioConductor.
The next step in building the list of dependencies and a script for installing
them is done in the following while loop. We start with a character vector
pkgs_to_download which is initially equivalent to OUR_PACKAGES. We will
iterate through this vector, appending the dependencies in order.
Use the tools::package_dependencies function to generate a list of the
packages dependencies, and dependencies of dependencies, and so on.
In the while loop we get a list of the dependencies for a package, stored in
the deps object. We will omit any of the base and recommended packages from
the deps object and then append deps to the pkgs_to_download vector in the
position immediately to the right of the current package being looked up. When
the indexer i is incremented, the next package to be considered will be the
first dependency. This process continues until all the dependencies have been
explored. Lastly, we reverse the order of the elements of pkgs_to_download
so that we have the packages listed in a useful install order, i.e.,
pkgs_to_download[1] should be installed before pkgs_to_download[2], etc.
After reversing the order of the elements of pkgs_to_download we look only at
the unique elements. By default, the first occurrence of an element will be
keep and the repeated elements will be omitted. By reversing the order then
taking the unique values, the deepest level of dependency will be retained for a
specific package.
<span class="n">pkgs_to_download</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">OUR_PACKAGES</span><span class="w">
</span><span class="n">i</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="m">1L</span><span class="w">
</span><span class="k">while</span><span class="p">(</span><span class="n">i</span><span class="w"> </span><span class="o"><=</span><span class="w"> </span><span class="nf">length</span><span class="p">(</span><span class="n">pkgs_to_download</span><span class="p">))</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="n">deps</span><span class="w"> </span><span class="o"><-</span><span class="w">
</span><span class="n">unlist</span><span class="p">(</span><span class="n">tools</span><span class="o">::</span><span class="n">package_dependencies</span><span class="p">(</span><span class="n">packages</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">pkgs_to_download</span><span class="p">[</span><span class="n">i</span><span class="p">],</span><span class="w">
</span><span class="n">which</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">c</span><span class="p">(</span><span class="s2">"Depends"</span><span class="p">,</span><span class="w"> </span><span class="s2">"Imports"</span><span class="p">,</span><span class="w"> </span><span class="s2">"LinkingTo"</span><span class="p">),</span><span class="w">
</span><span class="n">db</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">available_pkgs</span><span class="p">,</span><span class="w">
</span><span class="n">recursive</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">FALSE</span><span class="p">),</span><span class="w">
</span><span class="n">use.names</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">FALSE</span><span class="p">)</span><span class="w">
</span><span class="n">deps</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">deps</span><span class="p">[</span><span class="o">!</span><span class="p">(</span><span class="n">deps</span><span class="w"> </span><span class="o">%in%</span><span class="w"> </span><span class="n">base_pkgs</span><span class="p">)]</span><span class="w">
</span><span class="n">pkgs_to_download</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">append</span><span class="p">(</span><span class="n">pkgs_to_download</span><span class="p">,</span><span class="w"> </span><span class="n">deps</span><span class="p">,</span><span class="w"> </span><span class="n">i</span><span class="p">)</span><span class="w">
</span><span class="n">i</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">i</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="m">1L</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="n">pkgs_to_download</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">unique</span><span class="p">(</span><span class="n">rev</span><span class="p">(</span><span class="n">pkgs_to_download</span><span class="p">))</span><span class="w">
</span>If you are having a difficult time envisioning what the above does, let’s look
at and example for the dplyr package. In this example we’ll print out the
list of dependencies at each step through the while loop. Note that packages
such as Rcpp will be assessed multiple times, but the final list will only
have Rcpp listed once.
<span class="n">dplyr_dependencies</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="s2">"dplyr"</span><span class="w">
</span><span class="n">i</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="m">1L</span><span class="w">
</span><span class="k">while</span><span class="p">(</span><span class="n">i</span><span class="w"> </span><span class="o"><=</span><span class="w"> </span><span class="nf">length</span><span class="p">(</span><span class="n">dplyr_dependencies</span><span class="p">))</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="n">cat</span><span class="p">(</span><span class="s2">"\ni ="</span><span class="p">,</span><span class="w"> </span><span class="n">i</span><span class="p">,</span><span class="w"> </span><span class="s2">"\nLooking up dependencies for"</span><span class="p">,</span><span class="w"> </span><span class="n">dplyr_dependencies</span><span class="p">[</span><span class="n">i</span><span class="p">],</span><span class="w"> </span><span class="s2">"\n"</span><span class="p">)</span><span class="w">
</span><span class="n">deps</span><span class="w"> </span><span class="o"><-</span><span class="w">
</span><span class="n">unlist</span><span class="p">(</span><span class="n">tools</span><span class="o">::</span><span class="n">package_dependencies</span><span class="p">(</span><span class="n">packages</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">dplyr_dependencies</span><span class="p">[</span><span class="n">i</span><span class="p">],</span><span class="w">
</span><span class="n">which</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">c</span><span class="p">(</span><span class="s2">"Depends"</span><span class="p">,</span><span class="w"> </span><span class="s2">"Imports"</span><span class="p">,</span><span class="w"> </span><span class="s2">"LinkingTo"</span><span class="p">),</span><span class="w">
</span><span class="n">db</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">available_pkgs</span><span class="p">,</span><span class="w">
</span><span class="n">recursive</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">FALSE</span><span class="p">),</span><span class="w">
</span><span class="n">use.names</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">FALSE</span><span class="p">)</span><span class="w">
</span><span class="n">deps</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">deps</span><span class="p">[</span><span class="o">!</span><span class="p">(</span><span class="n">deps</span><span class="w"> </span><span class="o">%in%</span><span class="w"> </span><span class="n">base_pkgs</span><span class="p">)]</span><span class="w">
</span><span class="n">dplyr_dependencies</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">append</span><span class="p">(</span><span class="n">dplyr_dependencies</span><span class="p">,</span><span class="w"> </span><span class="n">deps</span><span class="p">,</span><span class="w"> </span><span class="n">i</span><span class="p">)</span><span class="w">
</span><span class="n">cat</span><span class="p">(</span><span class="n">dplyr_dependencies</span><span class="p">[</span><span class="n">i</span><span class="p">],</span><span class="w"> </span><span class="s2">"dependencies:"</span><span class="p">,</span><span class="w"> </span><span class="n">paste</span><span class="p">(</span><span class="n">deps</span><span class="p">,</span><span class="w"> </span><span class="n">collapse</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">", "</span><span class="p">),</span><span class="w">
</span><span class="s2">"\ndplyr_dependencies ="</span><span class="p">,</span><span class="w"> </span><span class="n">paste</span><span class="p">(</span><span class="n">dplyr_dependencies</span><span class="p">,</span><span class="w"> </span><span class="n">collapse</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">", "</span><span class="p">),</span><span class="w"> </span><span class="s2">"\n"</span><span class="p">)</span><span class="w">
</span><span class="n">i</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">i</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="m">1L</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="c1">## </span><span class="w">
</span><span class="c1">## i = 1 </span><span class="w">
</span><span class="c1">## Looking up dependencies for dplyr </span><span class="w">
</span><span class="c1">## dplyr dependencies: assertthat, bindrcpp, glue, magrittr, pkgconfig, rlang, R6, Rcpp, tibble, BH, plogr </span><span class="w">
</span><span class="c1">## dplyr_dependencies = dplyr, assertthat, bindrcpp, glue, magrittr, pkgconfig, rlang, R6, Rcpp, tibble, BH, plogr </span><span class="w">
</span><span class="c1">## </span><span class="w">
</span><span class="c1">## i = 2 </span><span class="w">
</span><span class="c1">## Looking up dependencies for assertthat </span><span class="w">
</span><span class="c1">## assertthat dependencies: </span><span class="w">
</span><span class="c1">## dplyr_dependencies = dplyr, assertthat, bindrcpp, glue, magrittr, pkgconfig, rlang, R6, Rcpp, tibble, BH, plogr </span><span class="w">
</span><span class="c1">## </span><span class="w">
</span><span class="c1">## i = 3 </span><span class="w">
</span><span class="c1">## Looking up dependencies for bindrcpp </span><span class="w">
</span><span class="c1">## bindrcpp dependencies: Rcpp, bindr, plogr </span><span class="w">
</span><span class="c1">## dplyr_dependencies = dplyr, assertthat, bindrcpp, Rcpp, bindr, plogr, glue, magrittr, pkgconfig, rlang, R6, Rcpp, tibble, BH, plogr </span><span class="w">
</span><span class="c1">## </span><span class="w">
</span><span class="c1">## i = 4 </span><span class="w">
</span><span class="c1">## Looking up dependencies for Rcpp </span><span class="w">
</span><span class="c1">## Rcpp dependencies: </span><span class="w">
</span><span class="c1">## dplyr_dependencies = dplyr, assertthat, bindrcpp, Rcpp, bindr, plogr, glue, magrittr, pkgconfig, rlang, R6, Rcpp, tibble, BH, plogr </span><span class="w">
</span><span class="c1">## </span><span class="w">
</span><span class="c1">## i = 5 </span><span class="w">
</span><span class="c1">## Looking up dependencies for bindr </span><span class="w">
</span><span class="c1">## bindr dependencies: </span><span class="w">
</span><span class="c1">## dplyr_dependencies = dplyr, assertthat, bindrcpp, Rcpp, bindr, plogr, glue, magrittr, pkgconfig, rlang, R6, Rcpp, tibble, BH, plogr </span><span class="w">
</span><span class="c1">## </span><span class="w">
</span><span class="c1">## i = 6 </span><span class="w">
</span><span class="c1">## Looking up dependencies for plogr </span><span class="w">
</span><span class="c1">## plogr dependencies: </span><span class="w">
</span><span class="c1">## dplyr_dependencies = dplyr, assertthat, bindrcpp, Rcpp, bindr, plogr, glue, magrittr, pkgconfig, rlang, R6, Rcpp, tibble, BH, plogr </span><span class="w">
</span><span class="c1">## </span><span class="w">
</span><span class="c1">## i = 7 </span><span class="w">
</span><span class="c1">## Looking up dependencies for glue </span><span class="w">
</span><span class="c1">## glue dependencies: </span><span class="w">
</span><span class="c1">## dplyr_dependencies = dplyr, assertthat, bindrcpp, Rcpp, bindr, plogr, glue, magrittr, pkgconfig, rlang, R6, Rcpp, tibble, BH, plogr </span><span class="w">
</span><span class="c1">## </span><span class="w">
</span><span class="c1">## i = 8 </span><span class="w">
</span><span class="c1">## Looking up dependencies for magrittr </span><span class="w">
</span><span class="c1">## magrittr dependencies: </span><span class="w">
</span><span class="c1">## dplyr_dependencies = dplyr, assertthat, bindrcpp, Rcpp, bindr, plogr, glue, magrittr, pkgconfig, rlang, R6, Rcpp, tibble, BH, plogr </span><span class="w">
</span><span class="c1">## </span><span class="w">
</span><span class="c1">## i = 9 </span><span class="w">
</span><span class="c1">## Looking up dependencies for pkgconfig </span><span class="w">
</span><span class="c1">## pkgconfig dependencies: </span><span class="w">
</span><span class="c1">## dplyr_dependencies = dplyr, assertthat, bindrcpp, Rcpp, bindr, plogr, glue, magrittr, pkgconfig, rlang, R6, Rcpp, tibble, BH, plogr </span><span class="w">
</span><span class="c1">## </span><span class="w">
</span><span class="c1">## i = 10 </span><span class="w">
</span><span class="c1">## Looking up dependencies for rlang </span><span class="w">
</span><span class="c1">## rlang dependencies: </span><span class="w">
</span><span class="c1">## dplyr_dependencies = dplyr, assertthat, bindrcpp, Rcpp, bindr, plogr, glue, magrittr, pkgconfig, rlang, R6, Rcpp, tibble, BH, plogr </span><span class="w">
</span><span class="c1">## </span><span class="w">
</span><span class="c1">## i = 11 </span><span class="w">
</span><span class="c1">## Looking up dependencies for R6 </span><span class="w">
</span><span class="c1">## R6 dependencies: </span><span class="w">
</span><span class="c1">## dplyr_dependencies = dplyr, assertthat, bindrcpp, Rcpp, bindr, plogr, glue, magrittr, pkgconfig, rlang, R6, Rcpp, tibble, BH, plogr </span><span class="w">
</span><span class="c1">## </span><span class="w">
</span><span class="c1">## i = 12 </span><span class="w">
</span><span class="c1">## Looking up dependencies for Rcpp </span><span class="w">
</span><span class="c1">## Rcpp dependencies: </span><span class="w">
</span><span class="c1">## dplyr_dependencies = dplyr, assertthat, bindrcpp, Rcpp, bindr, plogr, glue, magrittr, pkgconfig, rlang, R6, Rcpp, tibble, BH, plogr </span><span class="w">
</span><span class="c1">## </span><span class="w">
</span><span class="c1">## i = 13 </span><span class="w">
</span><span class="c1">## Looking up dependencies for tibble </span><span class="w">
</span><span class="c1">## tibble dependencies: cli, crayon, pillar, rlang </span><span class="w">
</span><span class="c1">## dplyr_dependencies = dplyr, assertthat, bindrcpp, Rcpp, bindr, plogr, glue, magrittr, pkgconfig, rlang, R6, Rcpp, tibble, cli, crayon, pillar, rlang, BH, plogr </span><span class="w">
</span><span class="c1">## </span><span class="w">
</span><span class="c1">## i = 14 </span><span class="w">
</span><span class="c1">## Looking up dependencies for cli </span><span class="w">
</span><span class="c1">## cli dependencies: assertthat, crayon </span><span class="w">
</span><span class="c1">## dplyr_dependencies = dplyr, assertthat, bindrcpp, Rcpp, bindr, plogr, glue, magrittr, pkgconfig, rlang, R6, Rcpp, tibble, cli, assertthat, crayon, crayon, pillar, rlang, BH, plogr </span><span class="w">
</span><span class="c1">## </span><span class="w">
</span><span class="c1">## i = 15 </span><span class="w">
</span><span class="c1">## Looking up dependencies for assertthat </span><span class="w">
</span><span class="c1">## assertthat dependencies: </span><span class="w">
</span><span class="c1">## dplyr_dependencies = dplyr, assertthat, bindrcpp, Rcpp, bindr, plogr, glue, magrittr, pkgconfig, rlang, R6, Rcpp, tibble, cli, assertthat, crayon, crayon, pillar, rlang, BH, plogr </span><span class="w">
</span><span class="c1">## </span><span class="w">
</span><span class="c1">## i = 16 </span><span class="w">
</span><span class="c1">## Looking up dependencies for crayon </span><span class="w">
</span><span class="c1">## crayon dependencies: </span><span class="w">
</span><span class="c1">## dplyr_dependencies = dplyr, assertthat, bindrcpp, Rcpp, bindr, plogr, glue, magrittr, pkgconfig, rlang, R6, Rcpp, tibble, cli, assertthat, crayon, crayon, pillar, rlang, BH, plogr </span><span class="w">
</span><span class="c1">## </span><span class="w">
</span><span class="c1">## i = 17 </span><span class="w">
</span><span class="c1">## Looking up dependencies for crayon </span><span class="w">
</span><span class="c1">## crayon dependencies: </span><span class="w">
</span><span class="c1">## dplyr_dependencies = dplyr, assertthat, bindrcpp, Rcpp, bindr, plogr, glue, magrittr, pkgconfig, rlang, R6, Rcpp, tibble, cli, assertthat, crayon, crayon, pillar, rlang, BH, plogr </span><span class="w">
</span><span class="c1">## </span><span class="w">
</span><span class="c1">## i = 18 </span><span class="w">
</span><span class="c1">## Looking up dependencies for pillar </span><span class="w">
</span><span class="c1">## pillar dependencies: cli, crayon, rlang, utf8 </span><span class="w">
</span><span class="c1">## dplyr_dependencies = dplyr, assertthat, bindrcpp, Rcpp, bindr, plogr, glue, magrittr, pkgconfig, rlang, R6, Rcpp, tibble, cli, assertthat, crayon, crayon, pillar, cli, crayon, rlang, utf8, rlang, BH, plogr </span><span class="w">
</span><span class="c1">## </span><span class="w">
</span><span class="c1">## i = 19 </span><span class="w">
</span><span class="c1">## Looking up dependencies for cli </span><span class="w">
</span><span class="c1">## cli dependencies: assertthat, crayon </span><span class="w">
</span><span class="c1">## dplyr_dependencies = dplyr, assertthat, bindrcpp, Rcpp, bindr, plogr, glue, magrittr, pkgconfig, rlang, R6, Rcpp, tibble, cli, assertthat, crayon, crayon, pillar, cli, assertthat, crayon, crayon, rlang, utf8, rlang, BH, plogr </span><span class="w">
</span><span class="c1">## </span><span class="w">
</span><span class="c1">## i = 20 </span><span class="w">
</span><span class="c1">## Looking up dependencies for assertthat </span><span class="w">
</span><span class="c1">## assertthat dependencies: </span><span class="w">
</span><span class="c1">## dplyr_dependencies = dplyr, assertthat, bindrcpp, Rcpp, bindr, plogr, glue, magrittr, pkgconfig, rlang, R6, Rcpp, tibble, cli, assertthat, crayon, crayon, pillar, cli, assertthat, crayon, crayon, rlang, utf8, rlang, BH, plogr </span><span class="w">
</span><span class="c1">## </span><span class="w">
</span><span class="c1">## i = 21 </span><span class="w">
</span><span class="c1">## Looking up dependencies for crayon </span><span class="w">
</span><span class="c1">## crayon dependencies: </span><span class="w">
</span><span class="c1">## dplyr_dependencies = dplyr, assertthat, bindrcpp, Rcpp, bindr, plogr, glue, magrittr, pkgconfig, rlang, R6, Rcpp, tibble, cli, assertthat, crayon, crayon, pillar, cli, assertthat, crayon, crayon, rlang, utf8, rlang, BH, plogr </span><span class="w">
</span><span class="c1">## </span><span class="w">
</span><span class="c1">## i = 22 </span><span class="w">
</span><span class="c1">## Looking up dependencies for crayon </span><span class="w">
</span><span class="c1">## crayon dependencies: </span><span class="w">
</span><span class="c1">## dplyr_dependencies = dplyr, assertthat, bindrcpp, Rcpp, bindr, plogr, glue, magrittr, pkgconfig, rlang, R6, Rcpp, tibble, cli, assertthat, crayon, crayon, pillar, cli, assertthat, crayon, crayon, rlang, utf8, rlang, BH, plogr </span><span class="w">
</span><span class="c1">## </span><span class="w">
</span><span class="c1">## i = 23 </span><span class="w">
</span><span class="c1">## Looking up dependencies for rlang </span><span class="w">
</span><span class="c1">## rlang dependencies: </span><span class="w">
</span><span class="c1">## dplyr_dependencies = dplyr, assertthat, bindrcpp, Rcpp, bindr, plogr, glue, magrittr, pkgconfig, rlang, R6, Rcpp, tibble, cli, assertthat, crayon, crayon, pillar, cli, assertthat, crayon, crayon, rlang, utf8, rlang, BH, plogr </span><span class="w">
</span><span class="c1">## </span><span class="w">
</span><span class="c1">## i = 24 </span><span class="w">
</span><span class="c1">## Looking up dependencies for utf8 </span><span class="w">
</span><span class="c1">## utf8 dependencies: </span><span class="w">
</span><span class="c1">## dplyr_dependencies = dplyr, assertthat, bindrcpp, Rcpp, bindr, plogr, glue, magrittr, pkgconfig, rlang, R6, Rcpp, tibble, cli, assertthat, crayon, crayon, pillar, cli, assertthat, crayon, crayon, rlang, utf8, rlang, BH, plogr </span><span class="w">
</span><span class="c1">## </span><span class="w">
</span><span class="c1">## i = 25 </span><span class="w">
</span><span class="c1">## Looking up dependencies for rlang </span><span class="w">
</span><span class="c1">## rlang dependencies: </span><span class="w">
</span><span class="c1">## dplyr_dependencies = dplyr, assertthat, bindrcpp, Rcpp, bindr, plogr, glue, magrittr, pkgconfig, rlang, R6, Rcpp, tibble, cli, assertthat, crayon, crayon, pillar, cli, assertthat, crayon, crayon, rlang, utf8, rlang, BH, plogr </span><span class="w">
</span><span class="c1">## </span><span class="w">
</span><span class="c1">## i = 26 </span><span class="w">
</span><span class="c1">## Looking up dependencies for BH </span><span class="w">
</span><span class="c1">## BH dependencies: </span><span class="w">
</span><span class="c1">## dplyr_dependencies = dplyr, assertthat, bindrcpp, Rcpp, bindr, plogr, glue, magrittr, pkgconfig, rlang, R6, Rcpp, tibble, cli, assertthat, crayon, crayon, pillar, cli, assertthat, crayon, crayon, rlang, utf8, rlang, BH, plogr </span><span class="w">
</span><span class="c1">## </span><span class="w">
</span><span class="c1">## i = 27 </span><span class="w">
</span><span class="c1">## Looking up dependencies for plogr </span><span class="w">
</span><span class="c1">## plogr dependencies: </span><span class="w">
</span><span class="c1">## dplyr_dependencies = dplyr, assertthat, bindrcpp, Rcpp, bindr, plogr, glue, magrittr, pkgconfig, rlang, R6, Rcpp, tibble, cli, assertthat, crayon, crayon, pillar, cli, assertthat, crayon, crayon, rlang, utf8, rlang, BH, plogr</span><span class="w">
</span><span class="n">dplyr_dependencies</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">unique</span><span class="p">(</span><span class="n">rev</span><span class="p">(</span><span class="n">dplyr_dependencies</span><span class="p">))</span><span class="w">
</span><span class="n">dplyr_dependencies</span><span class="w">
</span><span class="c1">## [1] "plogr" "BH" "rlang" "utf8" "crayon" </span><span class="w">
</span><span class="c1">## [6] "assertthat" "cli" "pillar" "tibble" "Rcpp" </span><span class="w">
</span><span class="c1">## [11] "R6" "pkgconfig" "magrittr" "glue" "bindr" </span><span class="w">
</span><span class="c1">## [16] "bindrcpp" "dplyr"</span><span class="w">
</span>Now that we have pkgs_to_download, a character vector of package names that
we need to download, we can use the download.packages function to do so. The
object dwnld_pkgs is a 2 column matrix with the name and file path to the
source file for each package.
<span class="c1"># Download the needed packages into the pkg-source-files directory</span><span class="w">
</span><span class="n">unlink</span><span class="p">(</span><span class="s2">"pkg-source-files/*"</span><span class="p">)</span><span class="w">
</span><span class="n">dir.create</span><span class="p">(</span><span class="s2">"pkg-source-files/"</span><span class="p">,</span><span class="w"> </span><span class="n">showWarnings</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">FALSE</span><span class="p">)</span><span class="w">
</span><span class="n">dwnld_pkgs</span><span class="w"> </span><span class="o"><-</span><span class="w">
</span><span class="n">download.packages</span><span class="p">(</span><span class="n">pkgs</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">pkgs_to_download</span><span class="p">,</span><span class="w">
</span><span class="n">destdir</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"pkg-source-files"</span><span class="p">,</span><span class="w">
</span><span class="n">repos</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">c</span><span class="p">(</span><span class="n">CRAN</span><span class="p">,</span><span class="w"> </span><span class="n">BIOC</span><span class="p">),</span><span class="w">
</span><span class="n">type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"source"</span><span class="p">)</span><span class="w">
</span><span class="n">head</span><span class="p">(</span><span class="n">dwnld_pkgs</span><span class="p">)</span><span class="w">
</span>The last step for the script to run on a machine with external http(s) request
authority, is to build a makefile to install all the needed packages. I
prefer the makefile over a bash script because of the default error handling
that a make provided compared to a bash script.
<span class="n">cat</span><span class="p">(</span><span class="s2">"all:\n"</span><span class="p">,</span><span class="w">
</span><span class="n">paste0</span><span class="p">(</span><span class="s2">"\tR CMD INSTALL "</span><span class="p">,</span><span class="w"> </span><span class="n">dwnld_pkgs</span><span class="p">[,</span><span class="w"> </span><span class="m">2</span><span class="p">],</span><span class="w"> </span><span class="s2">"\n"</span><span class="p">),</span><span class="w">
</span><span class="n">sep</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">""</span><span class="p">,</span><span class="w">
</span><span class="n">file</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"makefile"</span><span class="p">)</span><span class="w">
</span>For this example, the first several lines of the makefile are:
all:
R CMD INSTALL pkg-source-files/magrittr_1.5.tar.gz
R CMD INSTALL pkg-source-files/BH_1.66.0-1.tar.gz
R CMD INSTALL pkg-source-files/withr_2.1.1.tar.gz
R CMD INSTALL pkg-source-files/Rcpp_0.12.15.tar.gz
R CMD INSTALL pkg-source-files/gdtools_0.1.6.tar.gz
R CMD INSTALL pkg-source-files/svglite_1.2.1.tar.gzNote that magrittr is the last package in the OUR_PACKAGES object and has no
dependencies, thus is the first package installed. The svglite package is the
second to last package in OUR_PACKAGES and it will be installed after the
dependencies BH, withr, Rcpp, and gdtools are installed.
Installing on the Remote Machine
Now that the source files have been downloaded and the makefile generated,
move the pkg-source-files directory and the makefile to the remote machine
and run the makefile. If the makefile fails, there might be some system
dependencies that need to be updated.
Download the script and/or contribute
The build-dependency-list.R file can be found on my
github page.
To leave a comment for the author, please follow the link and comment on their blog: Peter E. DeWitt.
R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.