Speed up npm install with a local registry to cache packages
January 17, 2016Nicolas Girault5 min read
Remember the last time you ran the
npm install command, and you had time to grab three coffees, learn Russian and read Dostoievski before you could do anything productive?
During the installation of a node app, the
npm install step is doubtless the most time consuming one. Assuming your network is somehow slow it can be endless... It's inefficient because this command will download packages that you might have already downloaded tens of times for this app or another.
The npm cache already saves approximately 30% of the installation time: when packages are in the npm cache, the initial metadata request sends the cached ETag for the package, and in the vast majority of cases, the registry will return a 304 and the package tarball won't need to be downloaded again. However, the number of HTTP requests is still the same so there is some time that can still be saved.
Although I do recognize the social gain of taking a coffee break with your teammates, I will tell you how to avoid taking a coffee each time you run
npm install thanks to Nexus!
Nexus is a a repository manager developed by Sonatype. In other words it can simulate the npm registry on your host. It supports other repositories such as RPM, Maven, Docker Registry...
The simplest way to use Nexus is to pull the official Docker image.
docker run -d --name nexus-data sonatype/nexus docker run -d -p 8081:8081 --name nexus --volumes-from nexus-data sonatype/nexus
It can take some time (2-3 minutes) for the service to launch in a new container. You can tail the log to determine once Nexus is ready:
docker logs -f nexus
Otherwise following the official installation guide would take approximately 30 minutes.
Here are the main steps for ubuntu 15.10 (easily adaptable for other OS):
# install Java 8 JRE sudo add-apt-repository ppa:webupd8team/java sudo apt-get update sudo apt-get install oracle-java8-installer # Download Nexus archive sudo mkdir -p /opt/sonatype sudo wget http://download.sonatype.com/nexus/oss/nexus-installer-3.0.0-m6-unix-archive.tar.gz /opt/sonatype sudo tar xvzf nexus-installer-3.0.0-m6-unix-archive.tar.gz --directory /opt/sonatype && rm nexus-installer-3.0.0-m6-unix-archive.tar.gz sudo ln -s /opt/sonatype/nexus-3.0.0-b2015110601 /opt/sonatype/nexus # Launch the Nexus server: /opt/sonatype/nexus/bin/nexus run
Then you can access the web interface here: http://localhost:8081/
Set up the Nexus server
The next steps are based on Nexus guide for npm.
this video shows how to do the steps below!
- Log in as 'admin' with the default password 'admin123' (you should change it as explained in the post install checklist)
Create a npmjs proxy:
- Create a new repository: Parameters > Repositories > Create repository
- Select "npm proxy" (the "npm hosted" allow to store private packages and is not in the scope of this article)
- Fill the form (name: "npm-proxy", remote storage: "https://registry.npmjs.org" check "Use the Nexus truststore", select "default" for the blobstore) and click on "create repository"
Create a repository group
- Then create a new repository "npm group"
- I named it "npm" and fill the simple form including the repository you just created
Tell npm to hit the Nexus repository
~/.npmrc file, replace the line by:
# add the URL of the repository registry = http://localhost:8081/content/groups/npm/
Now try to install a package. Then you'll find it in the list of components in the nexus npm repository. You can disconnect from the Internet, remove your node modules and re-install. Voilà!
Unfortunately the official docker image does not provide the version 3 of Nexus so you cannot browse the npm packages.
Bonus: configure your local repository as a service
To avoid starting the nexus server each time you reboot, you can configure nexus as a service that will start during the boot phase.
If you're using a docker image, you'll just have to use the
--restart=always option in your
Otherwise, the Nexus guide explain how to do it but it is not simple and the doc is not up to date (this is actually the reason why I started using Docker).
There are alternatives to Nexus to cache npm packages. Here are two interesting ones:
- npm-cache wraps npm and includes cache utilities but it breaks the standard way to install node apps. Who expects to run
npm-cache installinstead of
npm installto install a node app?
- npm-proxy-cache is a simple node app doing the same job as Nexus but I don't like the fact it's not a formal registry (have you used it? Your feedback would be great).
Using Nexus saves approximately 30% of the installation time of packages already in the npm cache.
This is one of the many use cases of Nexus. You can easily set up this tool to store private packages and thus improve your installation process. This tool can be useful for companies with security concerns that do not want to access the Internet during the installation process.
So install Nexus but don't forget to heavily test your app to run
npm test and get another excuse to take a coffee break with your teammates!