If you experience performance issues with your Node.js application, you might be interested in trying the built-in Node profiler. In this article, I’ll show you how quickly you can get clues about the slow parts of your application and how it helped me to fix performance issues on my own project.
The built-in Node profiler lets you easily spot the blocking part of your code.
node --prof --log_timer_events path/to/your/app/entry/point/index.js node --prof-process --preprocess -j isolate-*.log > v8.json
Upload the JSON file on Flamebearer and you are good to go.
I’m building a Node.js application (based on the Nest.js Framework, thanks to Alexandre’s recommendations) which is intensely scraping web pages.
To be able to serve my users, I need to scrape roughly 250 pages per minute.
But at first, my program couldn’t scrape more than 50 pages / min and I was unable to find what was the bottleneck in my code. Instead of moving to a more powerful server, I decided to leverage the power of the Node V8 profiler and spot the culprit in my code. Here is how I used it.
How I solved it
Using the Node profiler
Run your application with the profiler:
node --prof --log_timer_events path/to/your/app/entry/point/index/index.js
Once your application has started, you can use Apache Bench to add load to your server by sending requests. This step is important because the profiler will measure the CPU time spent in each function, including the functions used to start the application. If you send only one request, you won’t see anything because the launch time of your app will be too long compared to your only one request.
To simulate 500 requests with a concurrency of 20, run this command:
ab -k -c 20 -n 500 "http://localhost:8000/"
-kEnable the HTTP KeepAlive feature (perform multiple requests within one HTTP session)
-c 20Set a concurrency of 20 requests
-n 500Send 500 requests
Once the requests are over, stop your application (
CTRL+C in the terminal).
During your program execution, the profiler logged which functions were running at every tick interval and wrote it in a log file.
You will now have a new tick file in your project directory (something like
Spotting the bottleneck
If you open the file, you will only find raw log lines from which it is, obviously, quite hard to get any insight.
So let’s process it by running:
node --prof-process --preprocess -j isolate-*.log > v8.json
It will convert the previous file into a (slightly) more readable JSON file.
But again, getting insight from the processed JSON file is a challenge. This is why we will use Flamebearer. It is a flame graph tool for Node and V8. I found it to be easier to use and more visual than other tools such as the Chrome DevTools Debugger.
You can install it locally by following the instructions on the repository but the online version is so easy to use you don’t want to bother installing anything locally. You just need to drag and drop the previously generated
v8.json file in Flamebearer. Once loaded, let’s focus on our scraper by clicking on its function:
Here, we can quickly see that the
cheerio.load() takes almost 80% of the CPU time.
From the Node profiler analysis, it is quite obvious that our performance problems are coming from Cheerio. Cheerio is a powerful library that allows us to parse and manipulate HTML DOM on server side with jQuery selectors. We really liked it but we realized that we would benefit from a lighter library and we decided to switch to the, perfectly named, node-fast-html-parser ⚡️. It’s not as powerful as Cheerio but it is much more faster! It’s basic but quite efficient. Here is the new generated flame graph after refactoring our code with this new library:
The scale is different from the previous screenshot, but we can see that the parsing takes approximately 60% of the CPU time.
The Node profiler is an efficient tool to help you improve the performances of your applications. It can help you save time by letting you easily find which functions are taking too much CPU time.
Happy debugging ☺️ !