How to Write an R Package Wrapping a NodeJS Module

[This article was first published on Colin Fay, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Mr A Bichat was looking for a way to
bundle a NodeJS module inside an R package. Here is an attempt at a
reproducible example, that might also help others!

About NodeJS Packages

There are two ways to install NodeJS packages: globally and locally.

The idea with local dependencies is that when writing your application
or your script, you bundle inside one big folder everything needed to
make that piece of JavaScript code run. That way, you can have various
versions of a Node module on the same computer without one interfering
with another. On a production server, that also means that when
publishing your app, you don’t have to care about machine-wide versions,
or about putting an app to prod with a version that might break another
application.

I love the way NodeJS allows to handle dependencies, but that’s the
subject for another day.

Node JS inside an R package

To create an app or cli in NodeJS, you will be following these steps:

  • Creating a new folder
  • Inside this folder, run npm init -y (the -y pre-fills all the
    fields) ; this function creates a package.json file
  • Create a JavaScript script (app.js, index.js, whatever.js)
    which will contain your JavaScript logic ; this file can take
    command lines arguments that will be processed inside the script
  • Install external modules with npm install modulename: this
    function adds elements to package.json, creates/add to
    package-lock.json, and the whole modulename and its deps are
    downloaded and put inside a node_modules/ folder inside your
    project

Once your software is built, be it an app or a cli, you will be sharing
to the world the package.json, package-lock.json, and all the files
that are required to run the tool. But not the node_modules/ folder,
which will be generated by the user.

Your soft can then be shared on npm, the Node package manager, shared
as a zip, or put on git, so that users can git clone the, and install
everything by running npm install inside the folder.

Let’s create a small example:

<span class="nb">cd</span> /tmp
<span class="nb">mkdir </span>nodeexample
<span class="nb">cd </span>nodeexample
npm init <span class="nt">-y</span>
Wrote to /private/tmp/nodeexample/package.json:

{
  "name": "nodeexample",
  "version": "1.0.0",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "chalk": "^4.0.0"
  },
  "devDependencies": {},
  "description": ""
}
<span class="nb">touch </span>whatever.js
npm <span class="nb">install </span>chalk
npm WARN [email protected] No description
npm WARN [email protected] No repository field.

+ [email protected]
updated 1 package and audited 7 packages in 6.686s

1 package is looking for funding
  run `npm fund` for details

found 0 vulnerabilities
<span class="nb">echo</span> <span class="s2">"const chalk = require('chalk');"</span> <span class="o">>></span> whatever.js
<span class="nb">echo</span> <span class="s2">"console.log(chalk.blue('Hello world'));"</span> <span class="o">>></span> whatever.js
<span class="nb">cat </span>whatever.js
const chalk = require('chalk');
console.log(chalk.blue('Hello world'));

Now this can be run with Node:

node /tmp/nodeexample/whatever.js
Hello world

Here is our current file structure:

<span class="n">fs</span><span class="o">::</span><span class="n">dir_tree</span><span class="p">(</span><span class="s2">"/tmp/nodeexample"</span><span class="p">)</span><span class="w">
</span>
/tmp/nodeexample
├── node_modules
│   ├── @types
│   │   └── color-name
│   │       ├── LICENSE
│   │       ├── README.md
│   │       ├── index.d.ts
│   │       └── package.json
│   ├── ansi-styles
│   │   ├── index.d.ts
│   │   ├── index.js
│   │   ├── license
│   │   ├── package.json
│   │   └── readme.md
│   ├── chalk
│   │   ├── index.d.ts
│   │   ├── license
│   │   ├── package.json
│   │   ├── readme.md
│   │   └── source
│   │       ├── index.js
│   │       ├── templates.js
│   │       └── util.js
│   ├── color-convert
│   │   ├── CHANGELOG.md
│   │   ├── LICENSE
│   │   ├── README.md
│   │   ├── conversions.js
│   │   ├── index.js
│   │   ├── package.json
│   │   └── route.js
│   ├── color-name
│   │   ├── LICENSE
│   │   ├── README.md
│   │   ├── index.js
│   │   └── package.json
│   ├── has-flag
│   │   ├── index.d.ts
│   │   ├── index.js
│   │   ├── license
│   │   ├── package.json
│   │   └── readme.md
│   └── supports-color
│       ├── browser.js
│       ├── index.js
│       ├── license
│       ├── package.json
│       └── readme.md
├── package-lock.json
├── package.json
└── whatever.js

As you can see, you have a node_modules folder that contains all the
modules, installed with your machine specific requirements.

Let’s now move this file to another folder (imagine it’s a git clone,
or you’ve received a zip), where we won’t be sharing the node_modules
folder: the users will have to install it to their machine.

<span class="nb">mkdir</span> /tmp/nodeexample2
<span class="nb">cp</span> /tmp/nodeexample/package-lock.json  /tmp/nodeexample2/package-lock.json
<span class="nb">cp</span> /tmp/nodeexample/package.json  /tmp/nodeexample2/package.json
<span class="nb">cp</span> /tmp/nodeexample/whatever.js  /tmp/nodeexample2/whatever.js

But if we try to run this script:

node /tmp/nodeexample2/whatever.js
node /tmp/nodeexample2/whatever.js
internal/modules/cjs/loader.js:979
  throw err;
  ^

Error: Cannot find module 'chalk'
Require stack:
- /private/tmp/nodeexample2/whatever.js
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:976:15)
    at Function.Module._load (internal/modules/cjs/loader.js:859:27)
    at Module.require (internal/modules/cjs/loader.js:1036:19)
    at require (internal/modules/cjs/helpers.js:72:18)
    at Object.<anonymous> (/private/tmp/nodeexample2/whatever.js:1:15)
    at Module._compile (internal/modules/cjs/loader.js:1147:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1167:10)
    at Module.load (internal/modules/cjs/loader.js:996:32)
    at Function.Module._load (internal/modules/cjs/loader.js:896:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [ '/private/tmp/nodeexample2/whatever.js' ]
}

We have a “Module not found” error: that’s because we haven’t installed
the dependencies yet. Let’s do that:

<span class="nb">cd</span> /tmp/nodeexample2 <span class="o">&&</span> npm <span class="nb">install</span>
npm WARN [email protected] No description
npm WARN [email protected] No repository field.

added 7 packages from 4 contributors and audited 7 packages in 2.132s

2 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
<span class="n">fs</span><span class="o">::</span><span class="n">dir_tree</span><span class="p">(</span><span class="s2">"/tmp/nodeexample2"</span><span class="p">,</span><span class="w"> </span><span class="n">recurse</span><span class="o">=</span><span class="w"> </span><span class="m">1</span><span class="p">)</span><span class="w">
</span>
/tmp/nodeexample2
├── node_modules
│   ├── @types
│   ├── ansi-styles
│   ├── chalk
│   ├── color-convert
│   ├── color-name
│   ├── has-flag
│   └── supports-color
├── package-lock.json
├── package.json
└── whatever.js
<span class="nb">cd</span> /tmp/nodeexample2 <span class="o">&&</span>  node whatever.js
Hello world

Tada 🎉!

Ok, but how can we bundle this into an R package? Here is how it will
work:

  • On our machine, we will create the full, working script into the
    inst/ folder of the package, and share everything but our
    node_modules folder
  • After the users have installed our package on their machines, they
    will have inside their package installation folder something that
    will look like the version of our /tmp/nodeexample2 just after our
    cp: script.js, package.json and package-lock.json (so no
    node_modules folder, hence no dependencies).
  • Then, from R, they will run an installation wrapper, that will call
    npm install inside the package installation folder, i.e inside
    system.file(package = "mypak"). That will add all the required
    node_modules.
  • Once the installation is completed, we will call the Node script
    inside the working directory where we just installed everything.
    This script will take command line arguments passed from R

node-minify

While I’m at it, let’s try to use something that I might use in the
future: node-minify, a node library which can minify CSS, notably
through the clean-css extension:
https://www.npmjs.com/package/@node-minify/clean-css.

If you don’t know what the minification is and what it’s used for, it’s
the process of removing every unnecessary characters from a file so that
it’s lighter. Because you know, on the web every byte counts.

See https://en.wikipedia.org/wiki/Minification_(programming) for more
info.

Step 1, create the package

I won’t expand on that, please refer to online documentation.

Step 2, initiate npm infrastructure

Once in the package, here is the script to initiate everything:

<span class="nb">mkdir</span> <span class="nt">-p</span> inst/node
<span class="nb">cd </span>inst/node 
npm init <span class="nt">-y</span>
npm <span class="nb">install</span> @node-minify/core @node-minify/clean-css

<span class="nb">touch </span>app.js

This app.js will do one thing: take the path to an input and an output
file, and then run the node-minify with these two arguments.

Step 3, creating the NodeJS script

Here is app.js:

<span class="kd">const</span> <span class="nx">minify</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">@node-minify/core</span><span class="dl">'</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">cleanCSS</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">@node-minify/clean-css</span><span class="dl">'</span><span class="p">);</span>

<span class="nx">minify</span><span class="p">({</span>
  <span class="na">compressor</span><span class="p">:</span> <span class="nx">cleanCSS</span><span class="p">,</span>
  <span class="na">input</span><span class="p">:</span> <span class="nx">process</span><span class="p">.</span><span class="nx">argv</span><span class="p">[</span><span class="mi">2</span><span class="p">],</span>
  <span class="na">output</span><span class="p">:</span> <span class="nx">process</span><span class="p">.</span><span class="nx">argv</span><span class="p">[</span><span class="mi">3</span><span class="p">],</span>
  <span class="na">callback</span><span class="p">:</span> <span class="p">(</span><span class="nx">e</span><span class="p">,</span> <span class="nx">res</span><span class="p">)</span> <span class="o">=></span> <span class="p">{}</span>
<span class="p">});</span>

Let’s now create a dummy css file:

<span class="nb">echo</span> <span class="s2">"body {"</span> <span class="o">>></span> test.css
<span class="nb">echo</span> <span class="s2">"  color:white;"</span> <span class="o">>></span> test.css
<span class="nb">echo</span> <span class="s2">"}"</span> <span class="o">>></span> test.css

And try to process it:

node app.js test.css test2.css
<span class="nb">cat </span>test2.css
body{color:#fff}

Nice, we now have a script in inst/ that can be run with Node! How to
make it available in R?

Step 4, building functions

Let’s start by ignoring the node_modules folder.

<span class="n">usethis</span><span class="o">::</span><span class="n">use_build_ignore</span><span class="p">(</span><span class="s2">"inst/node/node_modules/"</span><span class="p">)</span><span class="w">
</span>

Then, create a function that will install the Node app on the users’
machines, i.e where the package is installed.

<span class="n">minifyr_npm_install</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="k">function</span><span class="p">(</span><span class="w">
  </span><span class="n">force</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">FALSE</span><span class="w">
</span><span class="p">){</span><span class="w">
  </span><span class="c1"># Prompt the users unless they bypass (we're installing stuff on their machine)</span><span class="w">
  </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="o">!</span><span class="n">force</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
    </span><span class="n">ok</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">yesno</span><span class="o">::</span><span class="n">yesno</span><span class="p">(</span><span class="s2">"This will install our app on your local library.
                       Are you ok with that? "</span><span class="p">)</span><span class="w">
  </span><span class="p">}</span><span class="w"> </span><span class="k">else</span><span class="w"> </span><span class="p">{</span><span class="w">
    </span><span class="n">ok</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
  </span><span class="p">}</span><span class="w">
  
  </span><span class="c1"># If user is ok, run npm install in the node folder in the package folder</span><span class="w">
  </span><span class="c1"># We should also check that the infra is not already there</span><span class="w">
  </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">ok</span><span class="p">){</span><span class="w">
    </span><span class="n">processx</span><span class="o">::</span><span class="n">run</span><span class="p">(</span><span class="w">
      </span><span class="n">command</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"npm"</span><span class="p">,</span><span class="w"> 
      </span><span class="n">args</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">c</span><span class="p">(</span><span class="s2">"install"</span><span class="p">),</span><span class="w"> 
      </span><span class="n">wd</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">system.file</span><span class="p">(</span><span class="s2">"node"</span><span class="p">,</span><span class="w"> </span><span class="n">package</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"minifyr"</span><span class="p">)</span><span class="w">
    </span><span class="p">)</span><span class="w">
  </span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span>

Let’s now build a function to run the minifyer:

<span class="n">minifyr_run</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="k">function</span><span class="p">(</span><span class="w">
  </span><span class="n">input</span><span class="p">,</span><span class="w">
  </span><span class="n">output</span><span class="w">
</span><span class="p">){</span><span class="w">
  </span><span class="c1"># We're taking the absolute path as we will move to another folder to </span><span class="w">
  </span><span class="c1"># execute the Node Script</span><span class="w">
  </span><span class="n">input</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">fs</span><span class="o">::</span><span class="n">path_abs</span><span class="p">(</span><span class="n">input</span><span class="p">)</span><span class="w">
  </span><span class="n">output</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">fs</span><span class="o">::</span><span class="n">path_abs</span><span class="p">(</span><span class="n">output</span><span class="p">)</span><span class="w">
  </span><span class="n">processx</span><span class="o">::</span><span class="n">run</span><span class="p">(</span><span class="w">
    </span><span class="n">command</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"node"</span><span class="p">,</span><span class="w">
    </span><span class="n">args</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">c</span><span class="p">(</span><span class="w">
      </span><span class="s2">"app.js"</span><span class="p">,</span><span class="w">
      </span><span class="n">input</span><span class="p">,</span><span class="w">
      </span><span class="n">output</span><span class="w">
    </span><span class="p">),</span><span class="w">
    </span><span class="n">wd</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">system.file</span><span class="p">(</span><span class="s2">"node"</span><span class="p">,</span><span class="w"> </span><span class="n">package</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"minifyr"</span><span class="p">)</span><span class="w">
  </span><span class="p">)</span><span class="w">
  </span><span class="nf">return</span><span class="p">(</span><span class="n">output</span><span class="p">)</span><span class="w">
</span><span class="p">}</span><span class="w">
</span>

And here it is!

And with some extra package infrastructure, we’ve got everything we need
🙂

Step 5, try on our machine

Let’s run the built package on our machine:

<span class="c1"># To do once</span><span class="w">
</span><span class="n">minifyr</span><span class="o">::</span><span class="n">minifyr_npm_install</span><span class="p">()</span><span class="w">
</span>

Then, if we have a look at our local package lib:

<span class="n">fs</span><span class="o">::</span><span class="n">dir_tree</span><span class="p">(</span><span class="w">
  </span><span class="n">system.file</span><span class="p">(</span><span class="w">
    </span><span class="s2">"node"</span><span class="p">,</span><span class="w">
    </span><span class="n">package</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"minifyr"</span><span class="w">
  </span><span class="p">),</span><span class="w"> 
  </span><span class="n">recurse</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">FALSE</span><span class="w">
</span><span class="p">)</span><span class="w">
</span>
/Library/Frameworks/R.framework/Versions/3.6/Resources/library/minifyr/node
├── app.js
├── node_modules
├── package-lock.json
└── package.json

Let’s try our function:

<span class="c"># Dummy CSS creation</span>
<span class="nb">echo</span> <span class="s2">"body {"</span> <span class="o">></span> test.css
<span class="nb">echo</span> <span class="s2">"  color:white;"</span> <span class="o">>></span> test.css
<span class="nb">echo</span> <span class="s2">"}"</span> <span class="o">>></span> test.css
<span class="nb">cat </span>test.css
body {
  color:white;
}
<span class="n">minifyr</span><span class="o">::</span><span class="n">minifyr_run</span><span class="p">(</span><span class="w">
  </span><span class="s2">"test.css"</span><span class="p">,</span><span class="w"> 
  </span><span class="s2">"test2.css"</span><span class="w">
</span><span class="p">)</span><span class="w">
</span>
<span class="nb">cat </span>test2.css
body{color:#fff}

Tada 🎉!

Result package at: https://github.com/ColinFay/minifyr

Step 6, one last thing

Of course, one cool thing would be to test that npm and Node are
installed on the user’s machine. We can do that by running the version
commands fornpm and node, and check if the results of system() are
either 0 or 127, 127 meaning that the command failed to run.

<span class="n">node_available</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="k">function</span><span class="p">(){</span><span class="w">
  </span><span class="n">test</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">suppressWarnings</span><span class="p">(</span><span class="w">
    </span><span class="n">system</span><span class="p">(</span><span class="w">
      </span><span class="s2">"npm -v"</span><span class="p">,</span><span class="w">
      </span><span class="n">ignore.stdout</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">TRUE</span><span class="p">,</span><span class="w">
      </span><span class="n">ignore.stderr</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
    </span><span class="p">)</span><span class="w">
  </span><span class="p">)</span><span class="w">
  </span><span class="n">attempt</span><span class="o">::</span><span class="n">warn_if</span><span class="p">(</span><span class="w">
    </span><span class="n">test</span><span class="p">,</span><span class="w"> 
    </span><span class="o">~</span><span class="w"> </span><span class="n">.x</span><span class="w"> </span><span class="o">!=</span><span class="w"> </span><span class="m">0</span><span class="p">,</span><span class="w"> 
    </span><span class="s2">"Error launching npm"</span><span class="w">
  </span><span class="p">)</span><span class="w">
    </span><span class="n">test</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">suppressWarnings</span><span class="p">(</span><span class="w">
    </span><span class="n">system</span><span class="p">(</span><span class="w">
      </span><span class="s2">"node -v"</span><span class="p">,</span><span class="w">
      </span><span class="n">ignore.stdout</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">TRUE</span><span class="p">,</span><span class="w">
      </span><span class="n">ignore.stderr</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">TRUE</span><span class="w">
    </span><span class="p">)</span><span class="w">
  </span><span class="p">)</span><span class="w">
  </span><span class="n">attempt</span><span class="o">::</span><span class="n">message_if</span><span class="p">(</span><span class="w">
    </span><span class="n">test</span><span class="p">,</span><span class="w"> 
    </span><span class="o">~</span><span class="w"> </span><span class="n">.x</span><span class="w"> </span><span class="o">!=</span><span class="w"> </span><span class="m">0</span><span class="p">,</span><span class="w">
    </span><span class="s2">"Error launching Node"</span><span class="w">
  </span><span class="p">)</span><span class="w">
</span><span class="p">}</span><span class="w">
</span>

To leave a comment for the author, please follow the link and comment on their blog: Colin Fay.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)