May 14, 2020Cyrielle Willerval9 min read
According to a report made by The Shift Project, the carbon footprint of the digital is estimated to exceed air travel with more than 4% of the total greenhouse gas emissions. This number is expected to double in the next five years!
Green IT principles aims to reduce the impact of the digital. As a developer, I wondered how these principles could be used for the web applications I worked on, and what could be improved.
I first looked at the definition of digital. It includes:
In this chart, the Shift Project shows that the energy consumption can be divided in two factors:
So how can we measure the impact of a web application on our user’s device?
The data centers represents 19% of the digital energy consumption and some improvements in efficiency and performance can be found. However, there is no easy way to measure the carbon footprint of the server-side of an application. Therefore, it is complex to know where to begin to reduce the impact!
At Theodo, we use an internal application to display the developers' current missions and availabilities. It is used by the sale representatives to see which developers are available and to staff them on new projects. One person has access to a CSV file online, and updates it several times a day. It is important to always have up-to-date data on the application in order to correctly staff the developers.
To update the application database, a script runs every 15 minutes: it imports the CSV file and parses it, then modifies the availability of the developers in the database. The file, which is quite large, is therefore imported 96 times a day. I had the intuition that it was consuming a lot of energy to run this script as many times, and I looked for a way to confirm or deny it.
I searched “How can I measure my server energy consumption” online and found out that simplest way was to plug a wattmeter into my computer, run the script locally and monitor the energy consumption. However, like most people, I don’t have a wattmeter at home, and I was not very keen on buying one for this unique usage.
I finally discovered a very useful tool: PowerAPI.
PowerAPI is an open-source project developed by the Spirals research group (University of Lille 1 and Inria) in order to estimate the power consumption of a software, without any external device like a wattmeter. The calculation is made based on the analysis of the consumption of various hardware components (for instance CPU, memory, disk…). The objective is simple: design less energy-consuming software by easily monitoring their power consumption.
The power meter created by PowerAPI is composed of two components: the sensor and the formula
As the sensor needs information from the Linux kernel, PowerAPI can only be used on Linux machines and not on a virtual machine.
To install PowerAPI, you need:
In my case, I decided to execute the project locally and deploy the MongoDB instance on my computer.
In the MongoDB database, I created a database (I decided to call it
power-data) and two collections:
You can either monitor the global power consumption of your machine, or monitor the power consumption of all Docker containers. As my project was using Vagrant, I decided to monitor the global power consumption of my computer. However, I recommend monitoring the Docker containers as it will reduce the “noise” created by your machine.
To install PowerAPI, the PowerAPI documentation will guide you. Once the MongoDB instance is created, the power meter can be deployed in two commands and a few minutes. Here’s a rapid recap, to measure the global power consumption:
Deploy the sensor: launch the first command to create the sensor Docker container.
docker run --privileged --net=host --name powerapi-sensor --privileged -td \ -v /sys:/sys \ -v /var/lib/docker/containers:/var/lib/docker/containers:ro \ -v /tmp/powerapi-sensor-reporting:/reporting powerapi/hwpc-sensor \ -n powerapi-sensor \ -r "mongodb" -U "mongodb://ADDR" -D "power-data" -C "sensor-data" \ -s "rapl" -o -e RAPL_ENERGY_PKG
Deploy the formula: launch the second command to create the formula Docker container.
docker run -td --net=host --name powerapi-formula powerapi/rapl-formula \ -s \ --input mongodb -u mongodb://ADDR -d power-data -c sensor-data \ --output mongodb -u mongodb://ADDR -d power-data -c formula-data
The power meter is now ready, with two running Docker containers: the sensor and the formula.
If both Dockers are running, the power estimation is stored in the database. To stop monitoring, stop the Dockers.
Once PowerAPI was deployed:
The formula had stored all my power consumption data in the database. Now, I needed a way to display and analyse it.
First, I decided to use MongoDB Charts to display my results.
MongoDB provided me a simple interface to display the results written by the formula. I can for instance filter by date, to create a graph for a precise duration. On this chart, I displayed the energy consumption of my application during 1 minute.
However, in order to extract more information from the results, I decided to extract the data as a CSV, which can be done:
--output csvat the end of the command to launch the formula Docker
Once I had the data, I started to analyze it.
The results were given in Watts. When dealing with electric consumption, it is easier to use Watt-hours: for instance, you need 1 kWh to bake a cake in an oven during 1 hour. It means that your oven consumed 1000 Watts during 1 hour.
|Duration (s)||Average Power (W)||Consumption (Wh)|
The import lasted in average 225.5 seconds, about 3min45.
If I subtract the noise of the computer, an import alone consumes 0.29Wh. Imports are made 96 times a day, i.e. 35 064 times a year. It means the imports use each year around 10 kWh.
A Danish association, Tomorrow, created an interactive map to see in real time the countries' electricity production and its impact in terms of CO2. Based on this website and on an 24-hours average, in France, 10 kWh corresponds to 3.5 kg of CO2 emitted, as our energy comes mostly from nuclear. In the United States, 10 kWh corresponds to 11 kg CO2.
If the server running my application was behaving exactly like my computer, it would use around 7 kWh a year.
To sum up, I estimated that the application was using 17 kWh each year, and 60% of this energy consumption is made by the imports. The rest is the energy consumes by the server, in the hypothesis of a physical server.
What can we usually do with 17 kWh? With 17 kWh, you can use your air-conditioning for 102 hours, cook 17 chickens in your oven or travel 8.5 km with a small electric car.
It is quite a lot for the server of a small web application.
I spoke with the developer and the person which was updating the CSV file, and we decided to:
Thanks to PowerAPI, I was able to estimate the impact of a costly feature on a physical server. I provided figures to help the project team make decisions to reduce the carbon footprint of the application.
However, this server runs on AWS, and PowerAPI cannot be used in a virtual environment in the cloud. In order to estimate the real impact, we need to find a methodology, using data from the cloud provider: for instance the amount of CPU used by the application.
I will try to see the real impact in the cloud of this feature in an upcoming article!