How I Improved my Node Performances by 5 with the Profiler

If you experience performance issues with your Node.js application, you might be interested in trying the built-in Node profiler. In this article, I’ll show you how quickly you can get clues about the slow parts of your application and how it helped me to fix performance issues on my own project.

tl;dr

The built-in Node profiler lets you easily spot the blocking part of your code.

node --prof --log_timer_events path/to/your/app/entry/point/index.js
node --prof-process --preprocess -j isolate-*.log > v8.json

Upload the JSON file on Flamebearer and you are good to go.

My problem

I’m building a Node.js application (based on the Nest.js Framework, thanks to Alexandre’s recommendations) which is intensely scraping web pages.
To be able to serve my users, I need to scrape roughly 250 pages per minute.
But at first, my program couldn’t scrape more than 50 pages / min and I was unable to find what was the bottleneck in my code. Instead of moving to a more powerful server, I decided to leverage the power of the Node V8 profiler and spot the culprit in my code. Here is how I used it.

How I solved it

Using the Node profiler

Run your application with the profiler:

node --prof --log_timer_events path/to/your/app/entry/point/index/index.js

Once your application has started, you can use Apache Bench to add load to your server by sending requests. This step is important because the profiler will measure the CPU time spent in each function, including the functions used to start the application. If you send only one request, you won’t see anything because the launch time of your app will be too long compared to your only one request.

To simulate 500 requests with a concurrency of 20, run this command:

ab -k -c 20 -n 500 "http://localhost:8000/"
  • -k Enable the HTTP KeepAlive feature (perform multiple requests within one HTTP session)
  • -c 20 Set a concurrency of 20 requests
  • -n 500 Send 500 requests

Once the requests are over, stop your application (CTRL+C in the terminal).

What happened?

During your program execution, the profiler logged which functions were running at every tick interval and wrote it in a log file.

You will now have a new tick file in your project directory (something like isolate-*.log).

Spotting the bottleneck

If you open the file, you will only find raw log lines from which it is, obviously, quite hard to get any insight.

So let’s process it by running:

node --prof-process --preprocess -j isolate-*.log > v8.json

It will convert the previous file into a (slightly) more readable JSON file.

But again, getting insight from the processed JSON file is a challenge. This is why we will use Flamebearer. It is a flame graph tool for Node and V8. I found it to be easier to use and more visual than other tools such as the Chrome DevTools Debugger.

You can install it locally by following the instructions on the repository but the online version is so easy to use you don’t want to bother installing anything locally. You just need to drag and drop the previously generated v8.json file in Flamebearer. Once loaded, let’s focus on our scraper by clicking on its function:

Flamebearer Flamegraph

Node Profiler Flame Graph when using Cheerio

Here, we can quickly see that the cheerio.load() takes almost 80% of the CPU time.

Take action

From the Node profiler analysis, it is quite obvious that our performance problems are coming from Cheerio. Cheerio is a powerful library that allows us to parse and manipulate HTML DOM on server side with jQuery selectors. We really liked it but we realized that we would benefit from a lighter library and we decided to switch to the, perfectly named, node-fast-html-parser ⚡️. It’s not as powerful as Cheerio but it is much more faster! It’s basic but quite efficient. Here is the new generated flame graph after refactoring our code with this new library:

Flame graph generated with Flamebearer - fast

The scale is different from the previous screenshot, but we can see that the parsing takes approximately 60% of the CPU time.

Conclusion

The Node profiler is an efficient tool to help you improve the performances of your applications. It can help you save time by letting you easily find which functions are taking too much CPU time.

It is just one of the numerous debugging tools you can use to improve your JavaScript application performances. Whether it be your IDE built-in tools (such as the WebStorm IDE memory and CPU profiling functionnalities) or the Chrome Developer Tools (let’s say, if you want to improve network speed) you have plenty of tools to improve your project performances.

Happy debugging ☺️ !

Sources


You liked this article? You'd probably be a good match for our ever-growing tech team at Theodo.

Join Us