Node and npm modules

Node.js opened the door for developers to build performant web servers using JavaScript.

The explosion ofCommonJS modules which followed, created a massive new ecosystem. Building a typical website today involves hundreds, if not thousands, of modules.

To publish a module, you setmodule.exports in your code, create apackage.json file, and runnpm publish.

To consume a module, you add a dependency to yourpackage.json file, runnpm install, and callrequire('module-name') from your code.

Modules can depend on other modules.

Npm moves module files between a central registry and the machines running Node.js.

ESM modules

In2015,import andexport statements were added to JavaScript. ESM module loading is now a built-in feature ofall major browsers (sorry IE.)

ESM removes the need for package.json files, and uses URLs instead of npm module names -- but it does not preclude those from being used with ESM, say in a Node.js context.

To publish an ESM module, useexport in your code, and make the file fetchable by URL.

To consume an ESM module, useimport { ... } from URL. SeeMDN for more details.

Usingimport instead ofrequire() allows ESM modules to be loaded independently, without running the code where they are used. A variant of theimport statement, is thedynamic import() function. This allows for modules to be loaded asynchronously at run-time.

ESM is the basis for exciting new developer tools likeSnowpack andVite.

So, why are most modules still published with CommonJS?

Even before ESM, developers could use npm modules in front-end code. Tools likebrowserify orwebpack bundle modules into a single script file, loadable by browsers.

On the server side, it has taken Node.js a few years to arrive atESM support. Unfortunately, the 2 standards are not fully interoperable.

Despite everyone's best intentions, theNode.js docs are unclear about what to do. For a deeper explanation, I recommendthis article by Dan Fabulich.

Here is a summary of some interop scenarios:

require() from default Node.js context

• require("CommonJS-module") -Yes ✅, this has always worked and is the default.
• require("ESM-module") -No ❌.
• require("Dual-ESM-CJS-module") -Yes ✅, but be careful with state.

import statement from Node.js ESM context - E.g. in a server.mjs file.

• import from "ESM-module" -Yes ✅.
• import default from "CommonJS-module" -Yes ✅.
• import { name } from "CommonJS-module" -No ❌, get default.name

Dynamic Import as a fallback

Node's inability to require() ESM modules prevents simple upgrades from CommonJS to ESM.

Publishingdual ESM-CJS packages is messy because it involveswrapping CommonJS modules in ESM. Writing a module using ESM and then wrapping it for CommonJS is not possible.

Fortunately,dynamic import() provides an alternative.

Dynamic import() works from the default Node.js context as well as from an ESM context. You can even import() CJS modules. The only gotcha is that it returns a promise, so it is not a drop-in replacement for require().

Here is an example showing require() and import() together.

I publishedshortscale v1 as CommonJS. Forv2 and later the module is only available as ESM. This means that later releases can no longer be loaded using Node.js require().

Thisfastify server loads both module versions from a CJS context.

// minimal fastify server based on:// https://www.fastify.io/docs/latest/Getting-Started/#your-first-serverconstfastify=require('fastify')({logger:true});fastify.register(async(fastify)=>{letshortscale_v1=require('shortscale-v1');letshortscale_v4=(awaitimport('shortscale-v4')).default;// e.g. http://localhost:3000/shortscale-v1?n=47fastify.get('/shortscale-v1',function(req,res){letnum=Number(req.query.n);letstr=''+shortscale_v1(num);res.send({num,str});});// e.g. http://localhost:3000/shortscale-v4?n=47fastify.get('/shortscale-v4',function(req,res){letnum=Number(req.query.n);letstr=''+shortscale_v4(num);res.send({num,str});});});// Run the server!fastify.listen(3000,function(err,address){if(err){fastify.log.error(err);process.exit(1);}fastify.log.info(`server listening on${address}`);});

For this demo,package.json installs both versions of shortscale.

{"name":"demo-fastify-esm","version":"1.0.0","description":"Demonstrate ESM dynamic import from non-ESM server","main":"server.js","scripts":{"start":"node server.js"},"author":"Jurgen Leschner","license":"MIT","dependencies":{"fastify":"^3.11.0","shortscale-v1":"npm:shortscale@^1.1.0","shortscale-v4":"npm:shortscale@^4.0.0"},"repository":{"type":"git","url":"https://github.com/jldec/demo-fastify-esm"}}

I plan to migrate my modules to ESM. Othermodule authors are too.