<![CDATA[Theodo]]>https://blog.theodo.comGatsbyJSFri, 08 Jan 2021 15:05:01 GMT<![CDATA[Why Rome Tools Isn't Ready to Replace Eslint, Webpack and Babel... Yet]]>/2020/12/rome-tools-not-ready-to-replace-eslint-yet//2020/12/rome-tools-not-ready-to-replace-eslint-yet/Fri, 18 Dec 2020 00:00:00 GMT<style> figcaption { text-rendering: optimizeLegibility; -webkit-font-smoothing: antialiased; word-break: break-word; box-sizing: inherit; font-family: "Lucida Grande", "Lucida Sans Unicode", "Lucida Sans", Geneva, Arial, sans-serif; font-weight: 300; font-size: 16px; line-height: 20px; color: rgba(117, 117, 117, 1); margin-right: auto; max-width: 728px; margin-top: 10px; margin-left: auto; text-align: center; display: block; } code { font-family: "SFMono-Regular", Consolas, "Liberation Mono", Menlo, Courier, monospace; line-height: normal; background: rgba(135,131,120,0.15); color: #EB5757; border-radius: 3px; padding: 0.2em 0.4em; font-size: 0.7rem; } </style> <h2>Introduction</h2> <p>While building your React, Angular, or Vue project, you may have wondered why you needed to set up and configure <a href="https://babeljs.io/">Babel</a>, <a href="https://webpack.js.org/">Webpack</a>, <a href="https://eslint.org/">ESLint</a>, and <a href="https://prettier.io/">Prettier</a> separately and tried to look for an existing bundled implementation of all the features these tools provide. That is when you might have stumbled upon <a href="https://rome.tools/">Rome Tools</a>. I will show you in this article some of the best features Rome provides compared to ESLint as well as how I tried migrating my ESLint project to Rome:</p> <h3>TLDR</h3> <p>Rome has great features including explicit diagnostics, provides a VSCode extension which makes it a promising tool. However, only the linter and formatter are available and stability is a major pain point. In addition to this limitation, the lack of configuration options can scare off developers.</p> <h2>The Rome philosophy</h2> <p>Rome's ambition is to unify tools that have previously worked as separate. It wants to have strong conventions with minimal configuration. It aspires to go further than <code>create-react-app</code> or <code>vue-cli</code> as these tools just include multiple other tools with their dedicated configuration files. <strong>Rome aims at providing a single frontend development toolchain for all the following features</strong>:</p> <ul> <li>Bundling</li> <li>Compiling</li> <li>Documentation Generation</li> <li>Formatting</li> <li>Linting</li> <li>Minification</li> <li>Testing</li> <li>Type Checking</li> </ul> <p>So far, only the linter is supported by Rome Tools and it was built to provide features that ESLint has been lacking for some time, that is displaying diagnostics. I've used ESLint in many projects and I've always been annoyed by how poor the error messages displayed are. Rome tries to tackle that and I find that the diagnostics displayed with <code>rome check</code> are really helpful to <strong>understand the error, how you can fix it, and what you can do to avoid it appearing in the future</strong>:</p> <figure> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 703px; "> <a class="gatsby-resp-image-link" href="/static/b36275ab5f7a9bc7e74afa5a830f8abf/242e2/rome-diagnostics.png" style="display: block" target="_blank" rel="noopener"> <span class="gatsby-resp-image-background-image" style="padding-bottom: 26.486486486486488%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAFCAYAAABFA8wzAAAACXBIWXMAAAsSAAALEgHS3X78AAAAx0lEQVQY043PwW6DMBBFUWNjAwVsAoYASQtp1ChR1EVVumj+/79uHdZVm8XRG41GTxphtGZoWsZ2y853dG6DD176kWncMwwDt9s3X8tC5RxVdVfhwvwbIYWgibNVZ4qgpE5CppaNztHGoGKNiCQi3D5AkJcFpbNcjxPn+ZnP05GPyxvL5cT768zUeg6dZ/Y1dWr+L0yTFBsK80KsShtRhLQuIs0ET/l9J1d5Ef1dKJXCbgeyfo9yHtP0q6TbocoapTRSxo++yw9OT3CV5p8kzQAAAABJRU5ErkJggg==&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Imports don&apos;t comply with the order import rule, an explicit diagnostic is given and a fix is suggested" title="Imports don&apos;t comply with the order import rule, an explicit diagnostic is given and a fix is suggested" src="/static/b36275ab5f7a9bc7e74afa5a830f8abf/242e2/rome-diagnostics.png" srcset="/static/b36275ab5f7a9bc7e74afa5a830f8abf/1d79a/rome-diagnostics.png 185w, /static/b36275ab5f7a9bc7e74afa5a830f8abf/1efb2/rome-diagnostics.png 370w, /static/b36275ab5f7a9bc7e74afa5a830f8abf/242e2/rome-diagnostics.png 703w" sizes="(max-width: 703px) 100vw, 703px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </a> </span> <figcaption>Rome does a great job at displaying diagnostics!</figcaption> </figure> <p>Rome also includes a watch mode, and review to allow you to apply safe fixes, comment to disable the rule, ignore the issue, etc... all of that in its comprehensive CLI:</p> <figure> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <a class="gatsby-resp-image-link" href="/static/9ef2e74295fe9489f608b8d606e39b42/50383/rome-cli.png" style="display: block" target="_blank" rel="noopener"> <span class="gatsby-resp-image-background-image" style="padding-bottom: 35.67567567567567%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAHCAYAAAAIy204AAAACXBIWXMAAAsTAAALEwEAmpwYAAAA/ElEQVQoz5WR2U7DMBBF7Sy249hjZ2ka0kIRUSgF1FZQCv//YxeTgJS30oejOzMazcoet1u8f5ywPx5w/vrE7uUZPIqQiHRECAEpJZI0QRzHF2GkNDZlg7tiMbKiEpXKUSuDe1ejch5lWcJaC631RZhIEjhjsb7pMDz06JolurpBWy9QkkOmFBhj/0eGlaw18IVH27ahuIGTGVSI84gjClxVMM00VL1E6gqIqoEM03HrwHICJw+mDbihiRDn1k+a2wBN/k8OFaOyenWLp9MZ/f6I/vCG9e4V1K4Qkx+TWZKGzmFK/kc0s+f+rzrnMAwDuq4bj09E41evWnPGNz+PnAlrPvTUAAAAAElFTkSuQmCC&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="The props are using the spread syntax which is forbidden by Rome. The CLI explains the issue, as well as why this rule exists, and offers mulltiple options to fix the error." title="The props are using the spread syntax which is forbidden by Rome. The CLI explains the issue, as well as why this rule exists, and offers mulltiple options to fix the error." src="/static/9ef2e74295fe9489f608b8d606e39b42/50383/rome-cli.png" srcset="/static/9ef2e74295fe9489f608b8d606e39b42/1d79a/rome-cli.png 185w, /static/9ef2e74295fe9489f608b8d606e39b42/1efb2/rome-cli.png 370w, /static/9ef2e74295fe9489f608b8d606e39b42/50383/rome-cli.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </a> </span> <figcaption>Diagnostics, advice, and fix options</figcaption> </figure> <p>Rome is also shipped with its file history. Say you apply an autofix on all your files but forgot to exclude a directory, you can easily recover your formatted files with <code>rome recover pop</code>. I find this command truly handy, especially when setting up your formatting or auto-fixing your issues.</p> <h2>Experimenting and migrating from ESLint</h2> <p>To build up a better judgment of Rome Tools, I took one of my existing projects that were using ESLint and tried migrating it to Rome which made me encounter most of the problems Rome has.</p> <p>The first thing I did was to add Rome to my project. <strong>Applying formatting is a cakewalk with Rome</strong>: running <code>rome check --apply</code> and Rome will format and apply safe fixes to your entire codebase. I then started working on fixing the linting errors and that's when Rome hit me hard. Most of my React components were written as:</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">const</span> <span class="token function-variable function">Component</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token parameter">props<span class="token operator">:</span> Props</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">;</span></code></pre></div> <p>Rome has a rule against this (<code>useFunctionDeclarations</code>). It requires that you use function declarations instead of constant declarations:</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">function</span> <span class="token function">Component</span><span class="token punctuation">(</span><span class="token parameter">props<span class="token operator">:</span> Props</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token punctuation">}</span></code></pre></div> <p>I'm not going to go into details but I find that first syntax handy. In most of my projects, it allows me, at a glance, to quickly distinguish utils functions from components that return JSX. So I decided I wanted to remove that rule and that's when I discovered that <strong>configuring your linting rules is impossible to do with Rome</strong>. The contributors <a href="https://github.com/rome/tools/issues/20">chose to go with a zero-configuration stance</a>. The only thing that can be configured is the paths you want to ignore in the configuration file:</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx">lint<span class="token operator">:</span> <span class="token punctuation">{</span> ignore<span class="token operator">:</span> <span class="token punctuation">[</span><span class="token string">"src/__mocks__/"</span><span class="token punctuation">]</span><span class="token punctuation">;</span> <span class="token punctuation">}</span></code></pre></div> <p>You can also suppress a rule on a specific line with an explanation but it's far from ideal as these comments will undermine actual explanations comments in your code. <strong>The lack of configuration was my first pain point while experimenting with Rome</strong>.</p> <p>I still tried to fix all my linting errors while working with Rome with the CLI. While it seems promising, there are yet too many errors that cannot be fixed. Hence, the review CLI tool can prove a little useless when <strong>the only option it's offering is to comment out the error</strong>.</p> <p>I also had the case where the tool would present me with the same error over and over again until I realized the suppression comment was just not working properly:</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">return</span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">Style.Animate</span></span> <span class="token attr-name">className</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>animation<span class="token punctuation">}</span></span><span class="token punctuation">></span></span><span class="token plain-text"> // rome-ignore lint/jsx/noPropSpreading // rome-ignore lint/jsx/noPropSpreading // rome-ignore lint/jsx/noPropSpreading // rome-ignore lint/jsx/noPropSpreading // rome-ignore lint/jsx/noPropSpreading // rome-ignore lint/jsx/noPropSpreading // rome-ignore lint/jsx/noPropSpreading // rome-ignore lint/jsx/noPropSpreading // rome-ignore lint/jsx/noPropSpreading // rome-ignore lint/jsx/noPropSpreading </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">BaseComponent</span></span> <span class="token spread"><span class="token punctuation">{</span><span class="token punctuation">...</span><span class="token attr-value">props</span><span class="token punctuation">}</span></span> <span class="token punctuation">/></span></span></code></pre></div> <p><strong>Rome offers an extension to use in one IDE: Visual Studio Code</strong> (there is no plan to bring the equivalent to Webstorm for the moment). I was really happy when I discovered that because I rely a lot on my IDE to apply formatting, quick fixes, read lint errors output, etc...</p> <p>Overall, I can say that the extension does also an excellent job at displaying diagnostics. The information that is available in the CLI is also available when hovering on an error:</p> <figure> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <a class="gatsby-resp-image-link" href="/static/296eb834450f023457122e54980f7ef4/50383/rome-diagnostics-vscode.png" style="display: block" target="_blank" rel="noopener"> <span class="gatsby-resp-image-background-image" style="padding-bottom: 15.675675675675677%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAADCAYAAACTWi8uAAAACXBIWXMAAAsTAAALEwEAmpwYAAAArElEQVQI103Oy26DMBBAUUOIcWxsIPgFJURJiJr+/wfeumkXXVwdzUgjjVhER5oyoe3Zt8x8j6TNMcYG7ydC9MQUWD7mX5fMGM7I0aH9GV1URTVYjkeB2IaG+9iyZIWfDdPzQdxf+DXz/LqwvzYenys+WcJsmaLD9gZj1Tvbazp3equtRFx1x60s/Nrir+XgkpDdiYMUmKFCu7pY0yiBNIL654tKUB3+qv9V5m9TBEazFomguQAAAABJRU5ErkJggg==&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="The diagnostics are available on hover in Visual Studio Code" title="The diagnostics are available on hover in Visual Studio Code" src="/static/296eb834450f023457122e54980f7ef4/50383/rome-diagnostics-vscode.png" srcset="/static/296eb834450f023457122e54980f7ef4/1d79a/rome-diagnostics-vscode.png 185w, /static/296eb834450f023457122e54980f7ef4/1efb2/rome-diagnostics-vscode.png 370w, /static/296eb834450f023457122e54980f7ef4/50383/rome-diagnostics-vscode.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </a> </span> <figcaption>Explicit diagnostics straight in VSCode</figcaption> </figure> <p>Most of the safe fixes, as well as auto-formatting, can also be applied straight from your IDE without having to use the CLI, which I greatly appreciated. There are a few exceptions to this rule, for instance, I was not able to format my <code>tsconfig.json</code> using the extension but could do it from the CLI.</p> <p>However, I've been struggling a lot with that extension as well as with the CLI while migrating from ESLint to Rome as <strong>they both appear to lack stability</strong>. I had more than 200 files to migrate to Rome and I ran into a huge amount of crashes while doing so which made my experience with Rome pretty painful. I find it discouraging to have to restart my IDE or the CLI review because I keep getting exceptions:</p> <div class="gatsby-highlight" data-language="text"><pre class="language-text"><code class="language-text">! Rome exited as this error could not be handled and resulted in a fatal error. Please report if necessary. ✖ Connection died</code></pre></div> <p>Looking at the number of issues <a href="https://github.com/rome/tools/issues?q=is%3Aissue+is%3Aopen+extension">related to the extension</a> or the CLI, we can tell that stability is a major area of improvement for Rome.</p> <h2>Conclusion</h2> <p>I find it amazing that projects such as Rome emerge to tackle the weaknesses of predominant tools such as ESLint and offer alternatives. Rome tries to do that and <strong>I can only encourage these kinds of initiatives</strong>.</p> <p>The fact that only the linter/formatter is available isn't a major issue in my opinion. Even if it means that Rome isn't a bundled implementation yet, it is a <a href="https://github.com/rome/tools/issues/20">choice they made</a> to have a part of Rome ready as quickly as possible. Nevertheless, it lacks both configurability and stability. I would consider that point a red flag due to how it slowed me down while migrating. Rome wants to replace all the other tools but, paradoxically, makes it harder by not providing configurability.</p> <p>Finding out why your code isn't working and how you can improve your development practices is something Rome definitely provides but even with that, <strong>I cannot recommend starting a project with Rome at this day</strong>. I'm hoping that the Rome developers will stabilize it, offer configurability and implement the rest of the features Rome wants to provide to offer a developer-friendly monolith.</p><![CDATA[Get started with Django and Jupyter Notebooks on VSCode in minutes]]>/2020/11/django-jupyter-vscode-setup//2020/11/django-jupyter-vscode-setup/Thu, 17 Dec 2020 00:00:00 GMT<style> .three-dots { text-align: center; } .three-dots span { width: 5px; height: 5px; border-radius: 50%; background: black; display: inline-block; margin: 30px; } .caption { display: flex; justify-content: center; text-align: center; font-size: 14px; margin: 15px; } </style> <p><em>This is a guide primarily for developers familiar with Django, looking to add a data analytics or data science element to their web development project.</em></p> <p><em>Don't have much time and comfortable with Jupyter? Click here to see the initializer file you should create + import into every python script you want to run: <a href="#final-code">Final Code</a></em></p> <p><em>Want to see an example data analytics script you can use for any Django project? See here: <a href="#example-script">Example Script</a></em></p> <div class="three-dots"> <span></span> <span></span> <span></span> </div> <h2>Data Science for Web Apps</h2> <p><strong>Django</strong> is a (very popular) Python-based framework that helps you build web apps frighteningly quickly. A quick Google/YouTube search for Django tutorials and you'll have an avalanche of content coming your way.</p> <p><strong>Jupyter Notebook</strong> is a tool that lets you build python scripts fast. It's especially popular for data science prototyping as it lets you break up your code and re-run parts of your script.</p> <h2>Why merge the two?</h2> <p>Today, data analytics is (and should be) the mantra behind every successful product owner's launch plan - and as a developers, providing guidance and infrastructure to enable this is extremely valuable. However, combining these two in one stack can lead to some interesting errors and so, I've written this article to help you cross these hurdles quicker than I did.</p> <h3>Prerequisites</h3> <p>At this point, you should have a Django development server setup in a virtual environment (if not, there's <a href="https://www.google.com/search?q=setup+a+django+project">plenty of tutorials</a> for that).  This setup uses MacOS/OX (though OS shouldn't matter) with VSCode.</p> <p>If you don't have Python already set up in VSCode with a good formatter and would like to, <a href="#black-formatter">click here</a>.</p> <h2>Configuring Django to run Python scripts without the server</h2> <p>Start by creating a folder inside your Django root directory named <code>python_functions</code>. All your python scripts can live in here.</p> <div class="image" style="margin:30px auto; width:110%; text-align:center;"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 259px; "> <a class="gatsby-resp-image-link" href="/static/8b905def8807ef4b3e47daa39f7d64da/a2ead/image6.png" style="display: block" target="_blank" rel="noopener"> <span class="gatsby-resp-image-background-image" style="padding-bottom: 189.1891891891892%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAmCAIAAABLWSz8AAAACXBIWXMAAAsSAAALEgHS3X78AAADE0lEQVRIx5VW2Y7aQBDkV3aXwwbf5xz2zNj4YI3NciQbKZHykP//iBQ20Uq7ETaosWxE0dXVPdXMLMvmPLlcvp1O5/f3d1zP50uSpKuVttkY6/XmTswMwySE1nW93RaIoijzfBsEgabp95FXsK6vkRxX23YZ4/ghpI3jeDTtFYy3aVqO44I8Qgjhuh4ewWhSZtt2PM+v6x1ex+MJjyh4FPmROQhCKTMQFkJmWT6F8w2MmgEGRqlsCHAGo6ngKIrbtmua/eHwhvvRPun6PzDyoOZMSQgG5hBsNKdpmgMYrXJY7H9vha5pT0/Pi8VyVDASesbGuAmGzFJKSpmUCpqBwgOCxTFBtVmWtW2Lm6qqJw3J0GfUGYYxJgwU5ovlcrmamtkwTcfxCCHIj6m2TMscGy+j5zVbY7Zth5PgreaXVv48bX9diq5K7/eZRv5NMMO0XNfljPm+Z0MAy7JMcxJttNu2bd8P9vsWN8/PL4uHah7ULssSIiNwntGwqUcSExaGUdM0eZ73+Apz9kBmDAbOxjBe8/lDrTJMIJEcTUbP8Tj1SKIl8A0MCWMM1QL/8jKfch4/aENtIWWSplIpqGU7Dj6/5t9MoM0ofzscq7Le1a+7euc53mqpaau1rt2jMNN03TJtL/KjktIm8UToq4hWCW9E2qkoI/BgVPEp7NC9gh0HWlEqVFpmPJdMCSoFSZM44VSlEWeuG36NICI4SjPOlCra/Pwnb3f735fy0lXfDowpRvqgcMX8c/C8aPdCFjPLcj0vjAlyZzwVRVWrLPf9EPaCNWAhrP9F//m1VVAVHtJ1HXwbTtC1HXoOJ/pa6qfoPcwNYPuvrxjQPXYVAsY01XqxbfwgHKYapo1RHbbXOBidoCRhLAFzsAU4TVPM2SQwzjPP3ZB4URhzznsnCjGwE2kbzQ8jr+N2f8QLCxrFw3onLXdd0z1RBzHPMxznbb+xFGz0NtsjmXXdZcqPKKBFWcKM8P8AlSPz0K0RwTimMxXoEwLrDvjh2nUHqHiH//C3woFC+B5KxYqEZnhH1zsCb7iT/C8B2bJloxDW3gAAAABJRU5ErkJggg==&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Screenshot of django root file structure with new folder python_functions.py and new file hello.py inside" title="Screenshot of django root file structure with new folder python_functions.py and new file hello.py inside" src="/static/8b905def8807ef4b3e47daa39f7d64da/a2ead/image6.png" srcset="/static/8b905def8807ef4b3e47daa39f7d64da/1d79a/image6.png 185w, /static/8b905def8807ef4b3e47daa39f7d64da/a2ead/image6.png 259w" sizes="(max-width: 259px) 100vw, 259px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </a> </span> </div> <p>At this point, if you try and import something from Django (and you don't have your development server running), you'll get an error that your settings are not configured. To see the error, type the following into <code>hello.py</code> and run your script:</p> <div class="gatsby-highlight" data-language="python"><pre class="language-python"><code class="language-python"> <span class="token keyword">from</span> django<span class="token punctuation">.</span>contrib<span class="token punctuation">.</span>auth<span class="token punctuation">.</span>models <span class="token keyword">import</span> User</code></pre></div> <p>You should see an error message that ends with:</p> <div class="gatsby-highlight" data-language="python"><pre class="language-python"><code class="language-python"> django<span class="token punctuation">.</span>core<span class="token punctuation">.</span>exceptions<span class="token punctuation">.</span>ImproperlyConfigured<span class="token punctuation">:</span> Requested setting INSTALLED_APPS<span class="token punctuation">,</span> but settings are <span class="token keyword">not</span> configured<span class="token punctuation">.</span> You must either define the environment variable DJANGO_SETTINGS_MODULE <span class="token keyword">or</span> call settings<span class="token punctuation">.</span>configure<span class="token punctuation">(</span><span class="token punctuation">)</span> before accessing settings<span class="token punctuation">.</span></code></pre></div> <p>This happens because without running <code>manage.py</code>, Django doesn't configure and run your project - and so doesn't know where to import modules from.</p> <p>To fix this, we'll need to firstly specify an environment variable to our settings file and then call the setup function. As we'll have to do this for every python script we write, let's do all the setup in a separate script, and import it into <code>hello.py</code>.</p> <h3>Creating an initialiser script</h3> <p>Create a new file in the <code>python_functions</code> directory called <code>django_initializer.py</code>. In this file, we first set an environment variable pointing to our <code>settings.py</code> file, then run the function to setup Django.</p> <p>But we're not done yet! As <code>django_initializer.py</code> isn't in the project's root directory, python won't recognise <code>"arcwebsite.settings"</code> as a legitimate path. To fix this, we need to manually add the project root directory to the python path.</p> <p>Altogether, here's what we've got in <code>django_initializer.py</code>:</p> <div class="gatsby-highlight" data-language="python"><pre class="language-python"><code class="language-python"> <span class="token comment"># A script that's needed to setup django if it's not already running on a server.</span> <span class="token comment"># Without this, you won't be able to import django modules</span> <span class="token keyword">import</span> sys<span class="token punctuation">,</span> os<span class="token punctuation">,</span> django <span class="token comment"># Find the project base directory</span> BASE_DIR <span class="token operator">=</span> os<span class="token punctuation">.</span>path<span class="token punctuation">.</span>dirname<span class="token punctuation">(</span>os<span class="token punctuation">.</span>path<span class="token punctuation">.</span>dirname<span class="token punctuation">(</span>os<span class="token punctuation">.</span>path<span class="token punctuation">.</span>abspath<span class="token punctuation">(</span>__file__<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token comment"># Add the project base directory to the sys.path</span> <span class="token comment"># This means the script will look in the base directory for any module imports</span> <span class="token comment"># Therefore you'll be able to import analysis.models etc</span> sys<span class="token punctuation">.</span>path<span class="token punctuation">.</span>insert<span class="token punctuation">(</span><span class="token number">0</span><span class="token punctuation">,</span> BASE_DIR<span class="token punctuation">)</span> <span class="token comment"># The DJANGO_SETTINGS_MODULE has to be set to allow us to access django imports</span> os<span class="token punctuation">.</span>environ<span class="token punctuation">.</span>setdefault<span class="token punctuation">(</span><span class="token string">"DJANGO_SETTINGS_MODULE"</span><span class="token punctuation">,</span> <span class="token string">"arcwebsite.settings"</span><span class="token punctuation">)</span> <span class="token comment"># This is for setting up django</span> django<span class="token punctuation">.</span>setup<span class="token punctuation">(</span><span class="token punctuation">)</span></code></pre></div> <p>Remember, this is separate to the actual file we want to run, as seen below:</p> <div class="image" style="margin:30px auto; width:100%; text-align:center;"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <a class="gatsby-resp-image-link" href="/static/4f4ad862f2fe36207772f21e8d8d0154/50383/image7.png" style="display: block" target="_blank" rel="noopener"> <span class="gatsby-resp-image-background-image" style="padding-bottom: 35.13513513513513%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAHCAIAAACHqfpvAAAACXBIWXMAAAsSAAALEgHS3X78AAAA70lEQVQY0z2QaZKEIAyFvcq0hC2yRcSluuff9P2PNE8s/bSQPPJIzPD9+7a2xphSyrUuMteSC0I8pUhtS5EiIimnEAOY8N4M+/Hets17NsZOzEVyitFZp9RISuEiPzltSRtFelQ0Qn/dDOu6LUuF2Vr7uvWxY4hCmjh6Dp4n770z1sL8APNaRZzzqIxY3WCvtU4SpeUsISTH4czShp60Yd+PduxFZubp5+YqrjXFBBuKwqI0KQtnb/vKGRr6fh8hJYe2cHUHGyLCj2BCV3geMOPLHehYT/Pn88t9YJByB6NHz8jHnB+llILNtc7zDOUfU0BGwx/qgycAAAAASUVORK5CYII=&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Side by side screenshot of hello.py (empty) and django_initialiser.py" title="Side by side screenshot of hello.py (empty) and django_initialiser.py" src="/static/4f4ad862f2fe36207772f21e8d8d0154/50383/image7.png" srcset="/static/4f4ad862f2fe36207772f21e8d8d0154/1d79a/image7.png 185w, /static/4f4ad862f2fe36207772f21e8d8d0154/1efb2/image7.png 370w, /static/4f4ad862f2fe36207772f21e8d8d0154/50383/image7.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </a> </span> <span style="font-size: 14px; color: #b3b3b3">Make sure the initialiser script is in the same folder as your actual script</span> </div> <p>Run <code>hello.py</code> once more, and you should see it work with no problems!</p> <p>Now, you've got a setup to run Python scripts to analyse your data, that can interact with your Django database even when your server isn't running. This is especially useful when prototyping on a local machine before deploying any scripts to production.</p> <p>The final step is to set up Jupyter Notebook in VSCode, to interact with your Django project.</p> <h2>Setting up Jupyter Notebook in VSCode for a Django Project</h2> <h3>Why Jupyter Notebook?</h3> <p><a href="https://jupyter.org/">Jupyter Notebook</a> is a tool by <a href="https://jupyter.org/about">Project Jupyter</a> to make data science modelling and prototyping easy. One of the main benefits is the ability to split large <a href="https://hackr.io/blog/procedural-programming">procedural programming</a> scripts into individual lines or blocks, written in 'cells', and run them one at a time.</p> <div class="image" style="margin:30px auto; width:100%; text-align:center;"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <a class="gatsby-resp-image-link" href="/static/bc6ba868f8dcca77c390174e46dd4791/50383/image8.png" style="display: block" target="_blank" rel="noopener"> <span class="gatsby-resp-image-background-image" style="padding-bottom: 45.4054054054054%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAJCAYAAAAywQxIAAAACXBIWXMAAAsSAAALEgHS3X78AAABXklEQVQoz6WScWujQBDF/f5fK5DEcL1/whE1azUmtUabNnF1d2c182609OA4jlK68GMGhjdvhp2geTpwnh+QqEdWjzn2aQaV5ZjyKNlDKYU0TbHdbpEkCaIoQlmWOJ/PqKpqjnVd8xQFDn48PCDbx5xkR2yTXMjwSx1w0QRVlAjDEIvFAuv1es43m81fhOF6ri+XS16tVghkAj4dC3hyrNsWt7c3TJE8oes79KafJ+m6HkQE7/0fhmGAMWae+Hq94vL6ysG0Tqs1pjcJqluDl7qCPj1Bl89oZRUrIuccrDX/4KyddUROTDsEcRxD63ZuODmW0qA4nvBcVjC9hXUESx7OD/9H6jSM6HuDYB/v8PNwhRkA4zzqlqDdHTeh83iH+FNEhksrDaPdjk/NDcMd7MXF0AgaWRzvcF9CmnaGgyzLQNYwvv/4Po4I5MtZqRRN0/DHXX2VSu5PbhFFUfBvBp6ooMfdw7QAAAAASUVORK5CYII=&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Screenshot of Hello World script in an empty Jupyter Notebook" title="Screenshot of Hello World script in an empty Jupyter Notebook" src="/static/bc6ba868f8dcca77c390174e46dd4791/50383/image8.png" srcset="/static/bc6ba868f8dcca77c390174e46dd4791/1d79a/image8.png 185w, /static/bc6ba868f8dcca77c390174e46dd4791/1efb2/image8.png 370w, /static/bc6ba868f8dcca77c390174e46dd4791/50383/image8.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </a> </span> <span class="caption">An example use of the cell-by-cell feature of a Jupyter Notebook</span> </div> <p>Running cells individually has a lot of benefits - for one, you can split your script into data collection, pre-processing, model training, and prediction steps, and run these segments individually. This can be especially useful when tweaking hyper-parameters on your models for example (or for any other instances you may want to re-run just one section of code from your script).</p> <p>Jupyter Notebooks can of course be run in the browser, rather than in VSCode, using the following command:</p> <div class="gatsby-highlight" data-language="sh"><pre class="language-sh"><code class="language-sh"> (myenv) $ python manage.py shell_plus --notebook</code></pre></div> <p>With this however, you wouldn't get the benefit of other VSCode extensions on your ipynb file, and easy access to your non-python files within the same code editor. Thus, setting up Jupyter inside VSCode makes our dev experience that little bit better.</p> <h3>Installing Jupyter Notebook in Django</h3> <p>So let's get started. First we'll need to install Jupyter and django-extensions. <a href="https://django-extensions.readthedocs.io/en/latest/#">django-extensions</a> is a collection of custom extensions for the Django Framework. It's often needed to support many other custom packages such as this one.</p> <p>Run this command in your terminal:</p> <div class="gatsby-highlight" data-language="python"><pre class="language-python"><code class="language-python"> <span class="token punctuation">(</span>myenv<span class="token punctuation">)</span> $ pip install jupyter django<span class="token operator">-</span>extensions</code></pre></div> <p>Next, add <code>django_extensions</code> to your <code>INSTALLED_APPS</code> in <code>settings.py</code> and restart the django server:</p> <div class="image" style="margin:30px auto; width:110%; text-align:center;"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 412px; "> <a class="gatsby-resp-image-link" href="/static/03ed5a28b259f7dac972042e28c3f07c/9e32a/image9.png" style="display: block" target="_blank" rel="noopener"> <span class="gatsby-resp-image-background-image" style="padding-bottom: 100%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAUCAIAAAAC64paAAAACXBIWXMAAAsSAAALEgHS3X78AAAB3UlEQVQ4y41TW27bMBDUReyQIkVRfD8lSpEtx3bbjyZF0Y8Cvf9FukICFChi2YsFQRAcDmd2t6KEDMOQUrLOhRghW85p0yCMcb0GuR0VVzbntMzPX5bDdTnO4zCm4LUKSkQjW9bQDbAIw3630117GcIpu+L0Mbsp2OuY5uQaxhC+SV91vq8xdkqeSzxmX7yeox29gQRw8eYQrVOCEPo5GGPklbiOcekBbOAJQD4HO0e3ZG+koIRC3AZr+XXKlxKnaE9DmJM99QF+DidWSXpbc6kxkpxP3oDgoGQ2KmqZjOztuumtFh3/VHfFpIaHaUspFEg0DV83cBNsWp3CYMjNalVMKLhjTTcNNifdJw0raxpS1xtF+lcq0AyuLH2cghu9XfrQiQ4BObkDXw2jGPWCnb18Le5bNhevfNtIghRB8IEtcOsHjXc/JP4z6N9Z/goc1p+ev7n2yva6Rhv8VRcKxU9ZsO8lvAR1ifqarWmpJLWk9T1mZQmYwyh8ERGyR/UTeEzIe9bbmhkXgDW6G4sxir+PAiUfedftAm5b2b30AZqxBMsY257E/9sTWh/Ap94fkoOGehy81jmsvZ3OQxiD7Th/GOwyGATMMJLrAFFKHo4KFAohgApgwPcY5Uf8BVoYvMijt3v4AAAAAElFTkSuQmCC&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Screenshot of settings.py file with django_extensions string text inside the INSTALLED_APPS section" title="Screenshot of settings.py file with django_extensions string text inside the INSTALLED_APPS section" src="/static/03ed5a28b259f7dac972042e28c3f07c/9e32a/image9.png" srcset="/static/03ed5a28b259f7dac972042e28c3f07c/1d79a/image9.png 185w, /static/03ed5a28b259f7dac972042e28c3f07c/1efb2/image9.png 370w, /static/03ed5a28b259f7dac972042e28c3f07c/9e32a/image9.png 412w" sizes="(max-width: 412px) 100vw, 412px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </a> </span> </div> <p>As we're already using VSCode with the Python extension, we can create a new Notebook quite easily. Simply bring up the command bar (cmd+shift+P) and select in 'Create New Blank Jupyter Notebook'.</p> <div class="image" style="margin:30px auto; width:100%; text-align:center;"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 710px; "> <a class="gatsby-resp-image-link" href="/static/77e66d4ee3cae67c642e4167346e9332/7131f/image10.png" style="display: block" target="_blank" rel="noopener"> <span class="gatsby-resp-image-background-image" style="padding-bottom: 52.43243243243243%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAKCAIAAAA7N+mxAAAACXBIWXMAAAsSAAALEgHS3X78AAABYklEQVQoz42R+26DIBTGfZNVqRZQvFAVLKK1VXuxty3LsmXv/x47xXbNkv6xL1/ICfA7h3xY6ZzHstb7j1AfwnLHq11SbrjexuWWLXqSr6lofdkZt0GxZWoI1OCS0HOnlhB5KnW9ucz7r3r47A+vYvPVHL/70zudl06QIyYQgzVDocTyQNQbLs7OFCPkWGHIeMK7VVPo1UIvi6oTUkkhRJb5lPw1HR1QOjWybGRHjMnVqWgvfddlqqE+m80wxvR++YmBRAhZk4kdhwHXe14d1aIgBFNyxTAm/4EnURR1Xc/Kk1zu+65dt61SKk0z34cH+uP6a2JaP+A4jg18XqyGddPkQmZZnucC7sF88AiMjRgLwTfYtm3YbeoqUX1S7uqqgsl1XWutocXYhfM5ITDzGlaS8CiKETIwMvI8D9J3XAIFpOUZza6C5IjrehDN6JerJrbtPGDHcZDxU8E5usv80a3+ATinbj+OxMXyAAAAAElFTkSuQmCC&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Screenshot of VSCode search bar [use cmd+shift+P] with &apos;Create new Blank Jupyter Notebook&apos; typed in" title="Screenshot of VSCode search bar [use cmd+shift+P] with &apos;Create new Blank Jupyter Notebook&apos; typed in" src="/static/77e66d4ee3cae67c642e4167346e9332/7131f/image10.png" srcset="/static/77e66d4ee3cae67c642e4167346e9332/1d79a/image10.png 185w, /static/77e66d4ee3cae67c642e4167346e9332/1efb2/image10.png 370w, /static/77e66d4ee3cae67c642e4167346e9332/7131f/image10.png 710w" sizes="(max-width: 710px) 100vw, 710px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </a> </span> </div> <p>After opening your first <code>.ipynb</code> file (the file extension for Jupyter Notebooks) you should get a VSCode warning prompting you to install an ipykernel. Click on 'install' when this pops up. Once installed, you won't need to do this again for any subsequent notebook files you create in the same virtual environment.</p> <p><em>NB: This will change how your shell looks when you run <code>python manage.py shell</code> on the command line - but unless you've manually customised the default Django shell to your liking, this shouldn't be a problem.</em></p> <div class="image" style="margin:30px auto; width:100%; text-align:center;"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <a class="gatsby-resp-image-link" href="/static/efb870d8496529e0b15d9c25e253ead4/50383/image11.png" style="display: block" target="_blank" rel="noopener"> <span class="gatsby-resp-image-background-image" style="padding-bottom: 43.78378378378379%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAJCAIAAAC9o5sfAAAACXBIWXMAAAsSAAALEgHS3X78AAABSElEQVQozz1QCW7DIBDkIW3TAjaYwzjGtx0nUS71SNL+/zMdg+TRajWwuzMLpGmavu8rX03Tblgw1nVTlr5p2qqqnCvWKIptQIlq6T2OJANUlklhjNFa42Stdc7lAUopznkSIMTSk+fOWIMRtBHcRpamQspFCTkSa3OYwApjqC5XSmUBSmnvPYFwGoCFMBMd4g1KUsp5nruum6YRI4wtW8QRbE8opVimrmq8vA0AGccRHbGEpbA/XgxDGQD/VAiYEQa1oBQ9I9CBSZBxGGALuQn/uUOaoHs4HHKjnTUk4QlnnFEeM6MsBl80k8IVra+7dui7odxWDhu4rda2nh9ldyJv9BWxYcgvK1+Dph8yz5RTKs+EEdIKoVMumfAdVYbc8udJ/RzF99k8zvoOvnu/rTFtrsfkfpHPS/Z7yf6u6hd8Tz/37HYUX//tYWBusY3bmwAAAABJRU5ErkJggg==&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Screenshot of empty .ipynb file open in VSCode" title="Screenshot of empty .ipynb file open in VSCode" src="/static/efb870d8496529e0b15d9c25e253ead4/50383/image11.png" srcset="/static/efb870d8496529e0b15d9c25e253ead4/1d79a/image11.png 185w, /static/efb870d8496529e0b15d9c25e253ead4/1efb2/image11.png 370w, /static/efb870d8496529e0b15d9c25e253ead4/50383/image11.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </a> </span> </div> <p>Now, copy over the contents of <code>hello.py</code> (the two imports) into the new <code>.ipynb</code> file, and press run - your code should run without any errors!</p> <p>Now let's try a simple queryset operation. Add a line to display the first user on your system (or an equivalent queryset operation):</p> <div class="gatsby-highlight" data-language="python"><pre class="language-python"><code class="language-python"> <span class="token keyword">print</span><span class="token punctuation">(</span>User<span class="token punctuation">.</span>objects<span class="token punctuation">.</span><span class="token builtin">all</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">)</span></code></pre></div> <p>There's an error!</p> <div class="image" style="margin:30px auto; width:100%; text-align:center;"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <a class="gatsby-resp-image-link" href="/static/08e5f551f1c53add6ba33ee82a13340d/50383/image12.png" style="display: block" target="_blank" rel="noopener"> <span class="gatsby-resp-image-background-image" style="padding-bottom: 62.70270270270271%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAANCAYAAACpUE5eAAAACXBIWXMAABYlAAAWJQFJUiTwAAACKUlEQVQ4y41TW1LjMBD0MSCWLev9sGUnDgkkQNXe/1K9LcFCUfux+9E1siz19PSMupRnlGVFnmdkrmNM8N4ToUXn/g/1/EyOLtqE637Fy+2Gy+WKp6cL1mVDyWtLYK1rF2r8F2KM6FKOVLgghtiyOOdgjIUz3wfrt7X2B3k99w3/TWi2hLSvmNeC6/WKMxXO8wIdA3QtnRdCCC2Z5UWtNLT+G4r71aLuYfN4SAYH0WM573i+v+L17Q37/QXn+w2mzAhE2Y5YCE2lvRAYhuEHBPeMMej6ZPG4BSinUeaA05rZoIBYqHzLUNlDBY3JKWjCawOpHYRyGEk0jmNDJa22dMJNeAwKiZJftw33suGZSs7rhvNnvNOG9+cbflX1bNyd1hyPJ8SUmm+aSUTffxD2HJMDS+pTRr/UaDFEizFRAdWN2UFRqd1mGMKWjLAUyJgx2EClFkIHDNojkKt73BccWGZPheHErJylctxwop8rv2eqVmxMz5L6UTAKiOrZKDHIiqlB8n6d4e4w0+SoISQ94AiFmQoywXIsO92P9TKNl/Sq4YPgI8rm48Akkv/a2ByiQe10b3jAk7h4lpbhTwV+L3D7hjG6ZsHkWR7ns2d5wkSW6r+6LEneCCU7J0mqDwLeKBy5eWKp2/kJFzaj0K9MtTNfjePwSnZUTRITFU1V4WeHK2HiuS5ykSl/IcI0teEMfDWuvWPXZuvPi6hrH8LnW/dNkVLqi7D+/w1kSHbjCBU35QAAAABJRU5ErkJggg==&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Screenshot of .ipynb file with cells run and SynchronousOnlyOperation error message" title="Screenshot of .ipynb file with cells run and SynchronousOnlyOperation error message" src="/static/08e5f551f1c53add6ba33ee82a13340d/50383/image12.png" srcset="/static/08e5f551f1c53add6ba33ee82a13340d/1d79a/image12.png 185w, /static/08e5f551f1c53add6ba33ee82a13340d/1efb2/image12.png 370w, /static/08e5f551f1c53add6ba33ee82a13340d/50383/image12.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </a> </span> <span class="caption">Running queryset operations with the default setup causes errors</span> </div> <p>On closer inspection, you should see the error message ends with:</p> <div class="gatsby-highlight" data-language="text"><pre class="language-text"><code class="language-text"> SynchronousOnlyOperation: You cannot call this from an async context - use a thread or sync_to_async.</code></pre></div> <h2>Why does this error occur?</h2> <p><em>Skip to the next section if you'd just like the fix!</em></p> <p>Django was originally built with the WSGI standard in mind - a standard for synchronous Python web servers, frameworks and applications. With the release of Django 3.0 came <a href="https://docs.djangoproject.com/en/dev/releases/3.0/#asgi-support">support for ASGI</a>, a standard for asynchronous applications, allowing developers to create applications designed to run in an async context.</p> <p>However, certain key parts of Django rely on a global state being maintained, which is not coroutine-aware (i.e. not able to run in an async context). The most prominent example of this is Django's ORM (object-relational mapper) - essentially, its queryset filtering features, which allows you to interact with an SQL database through Python.</p> <p>Running any of these sync-only Django functions in an async context gives you the <strong><code>SynchronousOnlyOperation</code></strong> error. The ideal way to deal with this is to wrap your function in Django's <a href="https://docs.djangoproject.com/en/3.1/topics/async/#asgiref.sync.sync_to_async">sync_ to _async()</a>, but for a Jupyter Notebook where the async context is forced onto us by the environment (not our code), we need to manually allow sync-only operations, by setting <code>DJANGO_ALLOW_ASYNC_UNSAFE</code> to <code>true</code>. The Django documentation also has a section explaining <a href="https://docs.djangoproject.com/en/3.1/topics/async/#envvar-DJANGO_ALLOW_ASYNC_UNSAFE">async safety</a> and has stated that <a href="https://docs.djangoproject.com/en/3.1/topics/async/#asynchronous-support">async support for the ORM</a> will be available in future releases.</p> <h2>Fixing the SynchronousOnlyOperation Error</h2> <p>The quickest way to fix this is by setting another environment variable. Go to <code>django_initializer.py</code> and add:</p> <div class="gatsby-highlight" data-language="python"><pre class="language-python"><code class="language-python"> os<span class="token punctuation">.</span>environ<span class="token punctuation">[</span><span class="token string">"DJANGO_ALLOW_ASYNC_UNSAFE"</span><span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token string">"true"</span></code></pre></div> <div id="final-code"></div> <p>Which means <code>django_initialiser.py</code> in its final state now looks like:</p> <div class="gatsby-highlight" data-language="py"><pre class="language-py"><code class="language-py"><span class="token comment"># A script that's needed to setup django if it's not already running on a server.</span> <span class="token comment"># Without this, you won't be able to import django modules</span> <span class="token keyword">import</span> sys<span class="token punctuation">,</span> os<span class="token punctuation">,</span> django <span class="token comment"># Find the project base directory</span> BASE_DIR <span class="token operator">=</span> os<span class="token punctuation">.</span>path<span class="token punctuation">.</span>dirname<span class="token punctuation">(</span>os<span class="token punctuation">.</span>path<span class="token punctuation">.</span>dirname<span class="token punctuation">(</span>os<span class="token punctuation">.</span>path<span class="token punctuation">.</span>abspath<span class="token punctuation">(</span>__file__<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token comment"># Add the project base directory to the sys.path</span> <span class="token comment"># This means the script will look in the base directory for any module imports</span> <span class="token comment"># Therefore you'll be able to import analysis.models etc</span> sys<span class="token punctuation">.</span>path<span class="token punctuation">.</span>insert<span class="token punctuation">(</span><span class="token number">0</span><span class="token punctuation">,</span> BASE_DIR<span class="token punctuation">)</span> <span class="token comment"># The DJANGO_SETTINGS_MODULE has to be set to allow us to access django imports</span> os<span class="token punctuation">.</span>environ<span class="token punctuation">.</span>setdefault<span class="token punctuation">(</span><span class="token string">"DJANGO_SETTINGS_MODULE"</span><span class="token punctuation">,</span> <span class="token string">"arcwebsite.settings"</span><span class="token punctuation">)</span> <span class="token comment"># Allow queryset filtering asynchronously when running in a Jupyter notebook</span> os<span class="token punctuation">.</span>environ<span class="token punctuation">[</span><span class="token string">"DJANGO_ALLOW_ASYNC_UNSAFE"</span><span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token string">"true"</span> <span class="token comment"># This is for setting up django</span> django<span class="token punctuation">.</span>setup<span class="token punctuation">(</span><span class="token punctuation">)</span></code></pre></div> <p>Finally, reload the window and run the ipynb script again. You should get a success!</p> <div class="image" style="margin:30px auto; width:100%; text-align:center;"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <a class="gatsby-resp-image-link" href="/static/349987615095d1e22b54ec9c038e08cb/50383/image14.png" style="display: block" target="_blank" rel="noopener"> <span class="gatsby-resp-image-background-image" style="padding-bottom: 34.5945945945946%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAHCAYAAAAIy204AAAACXBIWXMAABYlAAAWJQFJUiTwAAAA5UlEQVQoz41R2Y6DMBDLfyzLkQtKIAcI0qb//13eTFRoK1WrPlj2TDQTO2FcNhBKYrhcIKXMUBBCouu6Bzg4F1+AlznG+x/ILGKISLeE6/WGfY/wLsAZh2maoXX/NRgXLUR2ZoxB3w/ngVIaSuri+D9Xhz6SsKr6zXEH3O+pONu2PWPLzqbizlp7srUOzrkHe4SwnHpdV8zzTAsrjOOIlGjhjmVZykK6IMZY+sTUCyGUIe990bScmGqao0tZXddQvYKZzBmVYr5G11oXpkd/QnysWdM0INB7tG1XQL976NfeO/jH+g+gkszV70BT+wAAAABJRU5ErkJggg==&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Screenshot of .ipynb file with cells run and no error message" title="Screenshot of .ipynb file with cells run and no error message" src="/static/349987615095d1e22b54ec9c038e08cb/50383/image14.png" srcset="/static/349987615095d1e22b54ec9c038e08cb/1d79a/image14.png 185w, /static/349987615095d1e22b54ec9c038e08cb/1efb2/image14.png 370w, /static/349987615095d1e22b54ec9c038e08cb/50383/image14.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </a> </span> </div> <h2>Finishing Touches</h2> <p>To recap - we've configured <strong>Django</strong> to give us access to the framework <strong>without the server running</strong> and set up a dev environment with <strong>Jupyter Notebooks</strong> for easy prototyping and testing of data analytics scripts.</p> <p>With these tools, you should now be able to run Python scripts inside of VSCode with familiar data science packages such as scikit-learn, pandas, and matplotlib, just as you would with a regular Jupyter Notebook - as well as have access to your Django database with queryset filtering.</p> <p>Good luck!</p> <div id="example-script"></div> <h2>Start Tracking Website Metrics - An Example</h2> <p>Here's an example script to track the total number of weekly user sign ups on your Django app from a given start date. Note that there are some dependencies required that haven't been mentioned in this tutorial, but after installing them, you should be good to go!</p> <div class="gatsby-highlight" data-language="py"><pre class="language-py"><code class="language-py"><span class="token comment"># Imports</span> <span class="token keyword">import</span> django_initializer <span class="token keyword">from</span> django<span class="token punctuation">.</span>contrib<span class="token punctuation">.</span>auth<span class="token punctuation">.</span>models <span class="token keyword">import</span> User <span class="token keyword">from</span> analysis<span class="token punctuation">.</span>models <span class="token keyword">import</span> WorkoutEntry <span class="token keyword">from</span> datetime <span class="token keyword">import</span> datetime<span class="token punctuation">,</span> timedelta <span class="token keyword">import</span> pytz <span class="token keyword">import</span> time <span class="token keyword">import</span> pandas <span class="token keyword">as</span> pd <span class="token keyword">import</span> numpy <span class="token keyword">as</span> np <span class="token keyword">import</span> matplotlib<span class="token punctuation">.</span>pyplot <span class="token keyword">as</span> plt pd<span class="token punctuation">.</span>set_option<span class="token punctuation">(</span><span class="token string">'display.max_rows'</span><span class="token punctuation">,</span> <span class="token number">500</span><span class="token punctuation">)</span> pd<span class="token punctuation">.</span>set_option<span class="token punctuation">(</span><span class="token string">'display.max_columns'</span><span class="token punctuation">,</span> <span class="token number">500</span><span class="token punctuation">)</span> pd<span class="token punctuation">.</span>set_option<span class="token punctuation">(</span><span class="token string">'display.width'</span><span class="token punctuation">,</span> <span class="token number">1000</span><span class="token punctuation">)</span> <span class="token comment"># Set a start date</span> start_date <span class="token operator">=</span> datetime<span class="token punctuation">(</span><span class="token number">2020</span><span class="token punctuation">,</span> <span class="token number">7</span><span class="token punctuation">,</span> <span class="token number">13</span><span class="token punctuation">,</span> tzinfo<span class="token operator">=</span>pytz<span class="token punctuation">.</span>UTC<span class="token punctuation">)</span> today <span class="token operator">=</span> pytz<span class="token punctuation">.</span>utc<span class="token punctuation">.</span>localize<span class="token punctuation">(</span>datetime<span class="token punctuation">.</span>today<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token comment"># Setup for weekly metrics</span> week_start <span class="token operator">=</span> start_date weeks <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token punctuation">]</span> <span class="token keyword">while</span> week_start <span class="token operator">&lt;=</span> today<span class="token punctuation">:</span> weeks<span class="token punctuation">.</span>append<span class="token punctuation">(</span>week_start<span class="token punctuation">)</span> week_start <span class="token operator">+=</span> timedelta<span class="token punctuation">(</span>days<span class="token operator">=</span><span class="token number">7</span><span class="token punctuation">)</span> week_strings <span class="token operator">=</span> <span class="token punctuation">[</span>week<span class="token punctuation">.</span>strftime<span class="token punctuation">(</span><span class="token string">"%d/%m"</span><span class="token punctuation">)</span> <span class="token keyword">for</span> week <span class="token keyword">in</span> weeks<span class="token punctuation">]</span> <span class="token comment"># Count number of cumulative sign ups</span> tot_users <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token punctuation">]</span> <span class="token comment"># total users</span> pc_growth <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token punctuation">]</span> <span class="token comment"># percentage growth week on week</span> <span class="token keyword">for</span> <span class="token punctuation">(</span>index<span class="token punctuation">,</span> week<span class="token punctuation">)</span> <span class="token keyword">in</span> <span class="token builtin">enumerate</span><span class="token punctuation">(</span>weeks<span class="token punctuation">)</span><span class="token punctuation">:</span> weekly_signups <span class="token operator">=</span> User<span class="token punctuation">.</span>objects<span class="token punctuation">.</span><span class="token builtin">filter</span><span class="token punctuation">(</span>is_active<span class="token operator">=</span><span class="token boolean">True</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token builtin">filter</span><span class="token punctuation">(</span>date_joined__gte<span class="token operator">=</span>start_date<span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token builtin">filter</span><span class="token punctuation">(</span>date_joined__lt<span class="token operator">=</span>weeks<span class="token punctuation">[</span>index<span class="token punctuation">]</span><span class="token punctuation">)</span> tot_users<span class="token punctuation">.</span>append<span class="token punctuation">(</span><span class="token builtin">len</span><span class="token punctuation">(</span>weekly_signups<span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token keyword">if</span> index <span class="token operator">></span> <span class="token number">1</span><span class="token punctuation">:</span> pc_growth<span class="token punctuation">.</span>append<span class="token punctuation">(</span>np<span class="token punctuation">.</span>around<span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token number">100</span><span class="token operator">*</span> <span class="token punctuation">(</span>tot_users<span class="token punctuation">[</span>index<span class="token punctuation">]</span> <span class="token operator">-</span> tot_users<span class="token punctuation">[</span>index<span class="token operator">-</span><span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token operator">/</span><span class="token punctuation">(</span>tot_users<span class="token punctuation">[</span>index<span class="token operator">-</span><span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token number">2</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token keyword">else</span><span class="token punctuation">:</span> pc_growth<span class="token punctuation">.</span>append<span class="token punctuation">(</span><span class="token number">0</span><span class="token punctuation">)</span> users_per_week_df <span class="token operator">=</span> pd<span class="token punctuation">.</span>DataFrame<span class="token punctuation">(</span><span class="token punctuation">{</span><span class="token string">"tot_users"</span><span class="token punctuation">:</span>tot_users<span class="token punctuation">,</span> <span class="token string">"pc_growth"</span><span class="token punctuation">:</span>pc_growth<span class="token punctuation">}</span><span class="token punctuation">,</span> index<span class="token operator">=</span>week_strings<span class="token punctuation">)</span> <span class="token keyword">print</span><span class="token punctuation">(</span>np<span class="token punctuation">.</span>mean<span class="token punctuation">(</span>np<span class="token punctuation">.</span>around<span class="token punctuation">(</span>pc_growth<span class="token punctuation">,</span> <span class="token number">2</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token keyword">print</span><span class="token punctuation">(</span>users_per_week_df<span class="token punctuation">)</span> fig<span class="token punctuation">,</span> axs <span class="token operator">=</span> plt<span class="token punctuation">.</span>subplots<span class="token punctuation">(</span>figsize<span class="token operator">=</span><span class="token punctuation">(</span><span class="token number">10</span><span class="token punctuation">,</span><span class="token number">6</span><span class="token punctuation">)</span><span class="token punctuation">)</span> axs<span class="token punctuation">.</span>plot<span class="token punctuation">(</span>users_per_week_df<span class="token punctuation">[</span><span class="token string">"tot_users"</span><span class="token punctuation">]</span><span class="token punctuation">)</span> axs<span class="token punctuation">.</span>set_xticks<span class="token punctuation">(</span>axs<span class="token punctuation">.</span>get_xticks<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token punctuation">:</span><span class="token punctuation">:</span><span class="token number">4</span><span class="token punctuation">]</span><span class="token punctuation">)</span> axs<span class="token punctuation">.</span>set_xlabel<span class="token punctuation">(</span><span class="token string">"Date"</span><span class="token punctuation">)</span> axs<span class="token punctuation">.</span>set_ylabel<span class="token punctuation">(</span><span class="token string">"Total Users"</span><span class="token punctuation">)</span> axs<span class="token punctuation">.</span>set_title<span class="token punctuation">(</span><span class="token string">"Total User Signups"</span><span class="token punctuation">,</span><span class="token punctuation">{</span><span class="token string">'fontsize'</span><span class="token punctuation">:</span> <span class="token number">18</span><span class="token punctuation">,</span> <span class="token string">'fontweight'</span><span class="token punctuation">:</span> <span class="token number">600</span><span class="token punctuation">}</span><span class="token punctuation">)</span> axs<span class="token punctuation">.</span>grid<span class="token punctuation">(</span><span class="token punctuation">)</span> plt<span class="token punctuation">.</span>show<span class="token punctuation">(</span><span class="token punctuation">)</span></code></pre></div> <div class="image" style="margin:30px auto; width:100%; text-align:center;"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <a class="gatsby-resp-image-link" href="/static/29f7c5589fddbadbc694359b28d046ae/50383/usergraph.png" style="display: block" target="_blank" rel="noopener"> <span class="gatsby-resp-image-background-image" style="padding-bottom: 63.78378378378379%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAANCAYAAACpUE5eAAAACXBIWXMAAAsSAAALEgHS3X78AAAB80lEQVQ4y62TS27bMBCGtQraVVY9QwAHuUFOUaC71vHWC++TtOgxfJAuvGrzKBK1RRAFjldGAReWjdpW89CLFMmhyM4wciAHSbvJAPT8Gg0//iPCnofR6/Ve9fv9rcFg0KAVBMHmeDze6Ha7261W62273X7T6XReU242mzuUfd/f+vVz2PDPgsbR9/PG1x8Xm2fBxQbxvNFotG8xAIDjUlJKRUGac66FELTKoiiAMWYLzg0+q5uMq4QVKs64TJgwjBeRAy4WCwfE0PRjjLHLeErnAqyAsqrfZa117ICTyeQ9FcqyVLgMmjP40tQ1bnJaKjApl+Y2L7BUGn3X44iYrx0wDMMPlQMgCI5Kp9EBmAE/hbZZoSxCLMKsUFQD954WalMBbxxwPp/vPQTi6VSxTEh7ywRCNG4ANx6N/hgQswOuIXC3GhloNGomF+QqRlfk1laQpfN/OsRL2a2+NQjQJkZHBFNAG/S9qyVsqSlXblccetPp1N0ylwAJk8aN5jaWdOqKq8c0XdaKw0kYuluOmRTYoEEpDdhNGpu0/r8Gcor5ygH/RNFH+wyBrqUDDofDd9Pfs8ssyz+nWXY4m82+pWl6mKHGC/OTJDnCf8yXKIpO4zg+zvP8AL/7KdUrfYL1E3z+5NViraZf1vSLWm39if57/Rc7T1IFuFsFBQAAAABJRU5ErkJggg==&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Graph showing total cumulative users from J" title="Graph showing total cumulative users from J" src="/static/29f7c5589fddbadbc694359b28d046ae/50383/usergraph.png" srcset="/static/29f7c5589fddbadbc694359b28d046ae/1d79a/usergraph.png 185w, /static/29f7c5589fddbadbc694359b28d046ae/1efb2/usergraph.png 370w, /static/29f7c5589fddbadbc694359b28d046ae/50383/usergraph.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </a> </span> </div> <div id="black-formatter"></div> <h2>Setting up Python in VSCode with the Black Formatter [optional]</h2> <p><em>We'll be adding the Python extension to VSCode, setting a virtual environment and setting up a formatter that you can customise to your preference.</em></p> <p>The first step is to ensure a great developer experience for programming in Python in VSCode. For this, you'll first need the VSCode Python extension, which you can find by simply searching for 'Python' in the extension sidebar. As of June 2020, there's also the new 'Pylance' extension, which is also suitable.</p> <p>Download this and reload the developer window (cmd+shift+P, type in 'reload' and you should see the command) and open a <code>.py</code> file. You should then see the Python version and interpreter in the bottom left corner of the screen. If it doesn't show your chosen virtual environment settings, click on it to update the settings to the interpreter of your choice (select from the dropdown or type in the path required).</p> <div class="image" style="margin:30px auto; width:100%; text-align:center;"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <a class="gatsby-resp-image-link" href="/static/b0a578e34004ce30114bbea572aa2887/50383/image2.png" style="display: block" target="_blank" rel="noopener"> <span class="gatsby-resp-image-background-image" style="padding-bottom: 51.35135135135135%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAKCAIAAAA7N+mxAAAACXBIWXMAAAsSAAALEgHS3X78AAABfUlEQVQoz42OW2+bMBhAeZiiwN6qLgumFLC5GLCxnczc07lQaFelatWHTdP+//+YSaVVqjptR0eyLX/HsgEAgBB92mxsG2gde6t1HXABtp7rnDb2iwDYn7cLtp4Dzmq1MgghN+NwdegYySnJ2+E4PvzspmdxuGfdHWtvXyy6b2zXcEpowbgQlBaWZRmXrlv1Yz89N9fHohxkN7TqjuzalFcJLWMqF4mMaJnifBrUNM11XQdBsF6vjcD32q+36v7H/PirG5/aplVXB8EFKyhn7GShD5wziuMdZ1LKJEkghKZpLnGv5n7+3k9P+2Ysy7rve0JokmCM0z8mOMuiEPme5/u6RAgtMQz8Xa1Ef5TqgVZzGC2v6kv4HuEJdGKJ9aL/FkI/QjDFcRRF6P9YYj/wZVnVTVuWVZrlYRhBGMC/gV41zbURQzRfj/Nw09YdowLHWYTiOEz+qWVaRnzmqVjVriTbPb+otNm5wGf8jem5KEBZAD32pXBkvtlbHz7+BoJ7YEgqHAjzAAAAAElFTkSuQmCC&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Screenshot of VSCode search bar, type command + shift + P to access" title="Screenshot of VSCode search bar, type command + shift + P to access" src="/static/b0a578e34004ce30114bbea572aa2887/50383/image2.png" srcset="/static/b0a578e34004ce30114bbea572aa2887/1d79a/image2.png 185w, /static/b0a578e34004ce30114bbea572aa2887/1efb2/image2.png 370w, /static/b0a578e34004ce30114bbea572aa2887/50383/image2.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </a> </span> </div> <p>Next, we'll set up the formatter - we're going to use <a href="https://black.readthedocs.io/en/stable/">Black</a> (<a href="https://github.com/psf/black">Github</a>). As Black is usually installed and run via the command line, we'll have to do some extra work to integrate it into VSCode, and run every time we save a file.</p> <p>There's no Black VSCode extension, so we'll have to install it the old fashioned way. In your terminal, run:</p> <div class="gatsby-highlight" data-language="sh"><pre class="language-sh"><code class="language-sh">(myenv) $ pip install black</code></pre></div> <p>Next, open your workspace settings (cmd+shift+P, type "settings", and select 'Workspace Settings' <strong>without</strong> JSON). In the search bar at the top, search 'format on save', then check the checkbox:</p> <div class="image" style="margin:30px auto; width:100%; text-align:center;"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <a class="gatsby-resp-image-link" href="/static/2f3c2710ab0dfce0f0020b433bb5c8d2/50383/image3.png" style="display: block" target="_blank" rel="noopener"> <span class="gatsby-resp-image-background-image" style="padding-bottom: 43.78378378378379%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAJCAIAAAC9o5sfAAAACXBIWXMAAAsSAAALEgHS3X78AAABF0lEQVQoz42RWW6DMBCGuUoYwAsYQ1hMWBqkREiueKDtS9WHHqD3P0B/TEk3Veqn0WjG9qz2zuPVnIbsWB8L81NKJzd71c3FPt0/vJxGS0TeqchVqsMgoDCiYCMEUMHuf0AUMdnY1/PjW3V9Jgq8tm211oeDb9Lkrmv7YSiKonQ0TYMrIYT8RDAWMmSmFa+qa7xAXsWiXKcgiiLm4JxDf3EZd4dcCJytwVIIOABZBsc4jlVVZd/Js0zpXKo8VkWaZkksfQRP02StRUxVln3fG2PQrdlBU8qBjhKVingVGHEs18rLsszzjNy4xvwYFX3JX7jJBQoiDLZbH3mo03UdloT0dV3D4H/A3LTbImhbmL9z+xH/H2zB73SyYAhq3zHXAAAAAElFTkSuQmCC&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Screenshot of VSCode workplace settings search bar, with &apos;format on save&apos; typed in, and &apos;user&apos; tab selected on the left" title="Screenshot of VSCode workplace settings search bar, with &apos;format on save&apos; typed in, and &apos;user&apos; tab selected on the left" src="/static/2f3c2710ab0dfce0f0020b433bb5c8d2/50383/image3.png" srcset="/static/2f3c2710ab0dfce0f0020b433bb5c8d2/1d79a/image3.png 185w, /static/2f3c2710ab0dfce0f0020b433bb5c8d2/1efb2/image3.png 370w, /static/2f3c2710ab0dfce0f0020b433bb5c8d2/50383/image3.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </a> </span> </div> <p>Finally, type in 'python formatting provider' and select 'black' from the dropdown:</p> <span class="image"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <a class="gatsby-resp-image-link" href="/static/310072971416d6bea9afbcf71093478b/50383/image4.png" style="display: block" target="_blank" rel="noopener"> <span class="gatsby-resp-image-background-image" style="padding-bottom: 30.270270270270274%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAGCAIAAABM9SnKAAAACXBIWXMAAAsSAAALEgHS3X78AAAAyklEQVQY03WPXQ6CMBCEuYq02KJ2+auRFpAALbyIiUZ98ATe/wQuCFET/dJNZtOdbseRUuYjKAAgGEER/gKvsqpr++surwkhDvZJkgghGGOcc3/G+2Y5k7T3/PSIzW0w4xyaUTHu141p6soaY63RSmmVaqw0VUrhDK7B5bD2xYptVmwwY1FKCXEJ9Ux/2bfHbdHuyi4tO2XP26wKA3h9GEbwgSCMBMBknqC0tAdZ2FDVkW4iXcWZ9UG6iwX5g/PZYGaOyd9niWnJf577D0E2kFq7vQAAAABJRU5ErkJggg==&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Screenshot of VSCode selecting black as default python formatter" title="Screenshot of VSCode selecting black as default python formatter" src="/static/310072971416d6bea9afbcf71093478b/50383/image4.png" srcset="/static/310072971416d6bea9afbcf71093478b/1d79a/image4.png 185w, /static/310072971416d6bea9afbcf71093478b/1efb2/image4.png 370w, /static/310072971416d6bea9afbcf71093478b/50383/image4.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </a> </span> </span> <p>Now reload your window, create a <code>.py</code> file and each time you save, you should see the words 'formatting with black' flicker on and off on the bottom information bar (in line with your python interpreter details). The next step is optional, and for those who want to customise Black.</p> <p>When running Black from the command line, you can pass in a <a href="https://github.com/psf/black">list of arguments</a> for custom preferences. To set these, go back into your VSCode settings and search 'black args'. You can then pass in individual strings. For example, to set maximum line length to 100 characters:</p> <div class="image" style="margin:30px auto; width:100%; text-align:center;"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <a class="gatsby-resp-image-link" href="/static/884a356d8bb876081348f83a9f373c20/50383/image5.png" style="display: block" target="_blank" rel="noopener"> <span class="gatsby-resp-image-background-image" style="padding-bottom: 28.10810810810811%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAGCAIAAABM9SnKAAAACXBIWXMAAAsSAAALEgHS3X78AAAArUlEQVQY043OTQ6CMBAF4F6F0j9oaZHUxlIpbthgTNR4CF14/60DjcgOv8UsJpn3BoUQnHO7LTWYp/PdcL6ehlEUEllrm6YRQlBKGWPLJCtpCUiOTbyExzvcXryyqJ7hLFOVbkPwbRtj7PseEueyiZSy+BKcM0o4JRCPUjbGWEjl/WFvrdbaGFOtwHH5IxN4B+UTAs1cmeP9WXej4Awayi3LcUogrFRMFPQ/8O8H8nQ+gDWQM/EAAAAASUVORK5CYII=&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Screenshot of VSCode preferences with &apos;black args&apos; typed into the search bar" title="Screenshot of VSCode preferences with &apos;black args&apos; typed into the search bar" src="/static/884a356d8bb876081348f83a9f373c20/50383/image5.png" srcset="/static/884a356d8bb876081348f83a9f373c20/1d79a/image5.png 185w, /static/884a356d8bb876081348f83a9f373c20/1efb2/image5.png 370w, /static/884a356d8bb876081348f83a9f373c20/50383/image5.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </a> </span> </div> <p>That's all you need - now your VSCode should be set up to code in Python, with automatic formatting on save.</p><![CDATA[Build your own stickers component with react-konva]]>/2020/11/react-konva-stickers//2020/11/react-konva-stickers/Mon, 16 Nov 2020 00:00:00 GMT<style> code { font-family: "SFMono-Regular", Consolas, "Liberation Mono", Menlo, Courier, monospace; line-height: normal; background: rgba(135,131,120,0.15); color: #EB5757; border-radius: 3px; padding: 0.2em 0.4em; font-size: 0.7rem; } </style> <p><a href="https://konvajs.org/" target="_blank">Konva.js</a> is an awesome HTML5 Canvas JavaScript framework that enables high performance animations for desktop and mobile applications.</p> <p>The framework's capabilities are useful and versatile - it can be integrated for many different and varied use cases. Konva.js deals well with <a href="https://konvajs.org/docs/sandbox/Gestures.html" target="_blank">touch genstures</a>, drag and drop and animations to name just a few!</p> <p>To see the breadth of the konva.js framework see the konva.js <a href="https://konvajs.org/docs/sandbox/index.html" target="_blank">demos page</a>.</p> <h3>React-Konva</h3> <p align="center"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 431px; "> <a class="gatsby-resp-image-link" href="/static/4f0c58ce52b4236652350104791ea31c/ee1a7/react-konva.jpg" style="display: block" target="_blank" rel="noopener"> <span class="gatsby-resp-image-background-image" style="padding-bottom: 67.02702702702703%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/jpeg;base64,/9j/2wBDABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVGC8aGi9jQjhCY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2P/wgARCAANABQDASIAAhEBAxEB/8QAFwABAQEBAAAAAAAAAAAAAAAAAAIDBf/EABUBAQEAAAAAAAAAAAAAAAAAAAAC/9oADAMBAAIQAxAAAAHuxrKbCv/EABcQAQEBAQAAAAAAAAAAAAAAAAECEBH/2gAIAQEAAQUCp4TXXCQz/8QAFhEAAwAAAAAAAAAAAAAAAAAAEBEh/9oACAEDAQE/Aax//8QAFBEBAAAAAAAAAAAAAAAAAAAAEP/aAAgBAgEBPwE//8QAGBAAAgMAAAAAAAAAAAAAAAAAEBEAASH/2gAIAQEABj8CcVnB/8QAGRABAQADAQAAAAAAAAAAAAAAAREAEDFB/9oACAEBAAE/Ia68xVCOkpHOSmv/2gAMAwEAAgADAAAAEMQP/8QAFxEBAQEBAAAAAAAAAAAAAAAAAREQIf/aAAgBAwEBPxBIR5n/xAAUEQEAAAAAAAAAAAAAAAAAAAAQ/9oACAECAQE/ED//xAAZEAEBAAMBAAAAAAAAAAAAAAABEQAQITH/2gAIAQEAAT8QC+gvMmlCk6JZoECiRMp+z3uv/9k=&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="react konva logo" title="react konva logo" src="/static/4f0c58ce52b4236652350104791ea31c/ee1a7/react-konva.jpg" srcset="/static/4f0c58ce52b4236652350104791ea31c/d7fe6/react-konva.jpg 185w, /static/4f0c58ce52b4236652350104791ea31c/f4308/react-konva.jpg 370w, /static/4f0c58ce52b4236652350104791ea31c/ee1a7/react-konva.jpg 431w" sizes="(max-width: 431px) 100vw, 431px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </a> </span> </p> <p><a href="https://github.com/konvajs/react-konva" target="_blank">React-Konva</a> allows us to work with both React applications and Konva.js together seamlessly.</p> <p>Here we'll create draggable stickers which can be deleted, suitable for both desktop and mobile applications. There are great react-native stickers libraries out there, however there really aren't many libraries to implement it in a react app. Therefore, react-konva is a great solution!</p> <h3>Show me the code!</h3> <p>Check out my code sandbox for the demo <a href="https://codesandbox.io/s/elated-shadow-ed4rj" target="_blank">here!</a></p> <h3>Let's get started...</h3> <h4>Adding a konva image</h4> <p>Let's take this tutorial through step-by-step. We need a canvas to add our stickers to. We'll add an image here to replicate a selfie, the sticker's most common use case.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">import</span> React <span class="token keyword">from</span> <span class="token string">"react"</span><span class="token punctuation">;</span> <span class="token keyword">import</span> <span class="token punctuation">{</span> Image <span class="token keyword">as</span> KonvaImage<span class="token punctuation">,</span> Layer<span class="token punctuation">,</span> Stage <span class="token punctuation">}</span> <span class="token keyword">from</span> <span class="token string">"react-konva"</span><span class="token punctuation">;</span> <span class="token keyword">import</span> useImage <span class="token keyword">from</span> <span class="token string">"use-image"</span><span class="token punctuation">;</span> <span class="token keyword">export</span> <span class="token keyword">default</span> <span class="token keyword">function</span> <span class="token function">App</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">const</span> <span class="token punctuation">[</span>image<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token function">useImage</span><span class="token punctuation">(</span><span class="token string">"example-image.png"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">return</span> <span class="token punctuation">(</span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">Stage</span></span> <span class="token attr-name">width</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span><span class="token number">500</span><span class="token punctuation">}</span></span> <span class="token attr-name">height</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span><span class="token number">500</span><span class="token punctuation">}</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">Layer</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">KonvaImage</span></span> <span class="token attr-name">image</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>image<span class="token punctuation">}</span></span> <span class="token punctuation">/></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span><span class="token class-name">Layer</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span><span class="token class-name">Stage</span></span><span class="token punctuation">></span></span> <span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span></code></pre></div> <p> <br> What is going on here?</p> <ul> <li>The <code>useImage</code> <a href="https://github.com/konvajs/use-image" target="_blank">hook</a> handles loading the image</li> <li>The <code>Stage</code> acts as our canvas</li> <li>The <code>Layer</code> is tied to the canvas element and can contain Groups, Shapes and Images</li> <li>The <code>KonvaImage</code> acts like an <code>img</code> tag and allows us to display our image</li> </ul> <p align="end"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 619px; "> <a class="gatsby-resp-image-link" href="/static/a29f08d13fe904f44370fd5f83130fe7/e628c/initial-image.png" style="display: block" target="_blank" rel="noopener"> <span class="gatsby-resp-image-background-image" style="padding-bottom: 68.10810810810811%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAAAsSAAALEgHS3X78AAADb0lEQVQ4y1WU6U5bZxCGfTW9gN5A1f5KryBKK0WVqkqV8qeVkqoNbSALgqYQgsGAwcYsBi94OzZJCnhjJ8aU4BSEIUBNsPFybOMNUqXt07FDI/XHaHS+b75nZt4ZHY3q83I2N0slFKC2EOZicYE3y4sNIxqBjUjD/7my1Di7WJznPByiGgxQlnfFp89QFQ+nTgelWAxN2jRMwWal5HJQUdzUphVqT7z8459lpu0eX3/6CUpLE/hnOJ/2UvMpVD0uKgIoWq2o42OkDUOcDAxQWFtFkzOPc+ZyUhZYVUDnT6f5KzjHa/sEtz+7gvH2da5//CFJm4W/A3ON+zq4KvElh52CdZLc2CinJhOl9XU0+ckJSm4nFa/nXfCzaZDWlx7/TP83VzmbHWD05jXWex5BOCgxPs59HqpuadFhew9Mj5goR6P/B9ae+KjJA0Qrpfk7Hn5xhazjEaZvr+Jvb5FEIYlRBOh+DywKUBVgZmSE8sbGf0DXpX7eBhQZkO6rawze/Jy4rokvP/oAy60bjfOq10VNkXjRvDRlpWCZQB0dITN6CcyNi4aOKSoed0PwC9GxOvsrC8ZfiDt7WWz/nvmhDlbMet4shRqwenVl0a8ouhYmzOQEljZdtlwvtWCzSflTEijZPU5K0VUqsRBxl4E99zDnOxHeJvZQN1c4k3bLshVFqSwvA23ADAZSYo2hpIeHyVssks1K2Tn1TpvIMpkFO5klL5WXAt9+ztn2bxRjEQpeN0VZFXVshKzJSNo4RFJW5rVeTzESQZMaHKS+OgWrRTSxUZFWsrM+UnMG0mHZzXiM2l4MdWOJwotlcooDddhIpg4z6EkOCqyvj0R/P4Xna2jqH/Xlro8+P2mmbLfzythFQunkUDGQ31yjFt/iaMbJH/Z2juQurR/kpL9PQDqOe3s56u7moKeH/KosdkIOUkaDjN2EKpWqZjMv9W3sTj4gOtDMqd9K1m8m6dFyaGtlT9fGiU5HQqsVkJaDrsfsd3YSF59bWUGze/8BBx0dJLq6SOoELrbd3Uaw6yc2jA/Zt2vZGrrLqvYHou23iLfeYb+1ld1799lpucvvd5rZavqRF+JTgQCabHie/PJyw4pScml1jdKmDODoFdXkMbXUMeXjQ053tkjLj0ENhciGw2SCIdKBIKcCSc35SYovHRzwL0chg5eXLV0fAAAAAElFTkSuQmCC&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Woman in a pink dress resting her hand under her chin" title="Woman in a pink dress resting her hand under her chin" src="/static/a29f08d13fe904f44370fd5f83130fe7/e628c/initial-image.png" srcset="/static/a29f08d13fe904f44370fd5f83130fe7/1d79a/initial-image.png 185w, /static/a29f08d13fe904f44370fd5f83130fe7/1efb2/initial-image.png 370w, /static/a29f08d13fe904f44370fd5f83130fe7/e628c/initial-image.png 619w" sizes="(max-width: 619px) 100vw, 619px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </a> </span> <span style="font-size:14px;margin-right:55px">Photo by Moose Photos from Pexels</span> </p> <p><strong>Aside:</strong> When I was building this for a photobooth experience I utilised the <a href="https://github.com/MABelanger/react-html5-camera-photo">react-html5-camera-photo</a> library to take the photo that we used as the image in the canvas, <em>but let's stick to the stickers for now</em> 👀</p> <h4>Creating the sticker button</h4> <p>Let's add a button with our desired sticker. We'll click this button whenever we want to add a new sticker to the canvas.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>button</span> <span class="token attr-name">className</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>button<span class="token punctuation">"</span></span> <span class="token attr-name">onMouseDown</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token function">addStickerToPanel</span><span class="token punctuation">(</span><span class="token punctuation">{</span> src<span class="token operator">:</span> <span class="token string">"sticker-1.png"</span><span class="token punctuation">,</span> width<span class="token operator">:</span> <span class="token number">100</span><span class="token punctuation">,</span> x<span class="token operator">:</span> <span class="token number">100</span><span class="token punctuation">,</span> y<span class="token operator">:</span> <span class="token number">100</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">}</span></span> <span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>img</span> <span class="token attr-name">alt</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>Eiffel Tower<span class="token punctuation">"</span></span> <span class="token attr-name">src</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>sticker-1.png<span class="token punctuation">"</span></span> <span class="token attr-name">width</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>100px<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>button</span><span class="token punctuation">></span></span></code></pre></div> <p>We use the button's <code>onMouseDown</code> property to add the sticker to our array of images to be added to our canvas. We set the width and the <code>x</code> and <code>y</code> position where the sticker should appear on the canvas.</p> <p>We initialise an empty array called <code>images</code>, using react's <code>useState</code> hook, and add the <code>width</code> and <code>src</code> properties to the array when we click a sticker!</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">const</span> <span class="token punctuation">[</span>images<span class="token punctuation">,</span> setImages<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token function">useState</span><span class="token punctuation">(</span><span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">const</span> <span class="token function-variable function">addStickerToPanel</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token parameter"><span class="token punctuation">{</span> src<span class="token punctuation">,</span> width<span class="token punctuation">,</span> x<span class="token punctuation">,</span> y <span class="token punctuation">}</span></span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token function">setImages</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token parameter">currentImages</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">[</span> <span class="token operator">...</span>currentImages<span class="token punctuation">,</span> <span class="token punctuation">{</span> width<span class="token punctuation">,</span> src<span class="token punctuation">,</span> x<span class="token punctuation">,</span> y <span class="token punctuation">}</span> <span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">;</span></code></pre></div> <h4>Adding a draggable sticker</h4> <p>Let's create a separate component that contains our individual stickers which will be on the canvas.</p> <p>We can only use konva components within our <code>Stage</code> in App.js so we use a Konva Image component.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">import</span> useImage <span class="token keyword">from</span> <span class="token string">"use-image"</span><span class="token punctuation">;</span> <span class="token keyword">import</span> React <span class="token keyword">from</span> <span class="token string">"react"</span><span class="token punctuation">;</span> <span class="token keyword">import</span> <span class="token punctuation">{</span> Image <span class="token keyword">as</span> KonvaImage <span class="token punctuation">}</span> <span class="token keyword">from</span> <span class="token string">"react-konva"</span><span class="token punctuation">;</span> <span class="token keyword">export</span> <span class="token keyword">const</span> <span class="token function-variable function">IndividualSticker</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token parameter"><span class="token punctuation">{</span> image <span class="token punctuation">}</span></span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">const</span> <span class="token punctuation">[</span>stickerImage<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token function">useImage</span><span class="token punctuation">(</span>image<span class="token punctuation">.</span>src<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">const</span> stickerHeight <span class="token operator">=</span> stickerImage <span class="token operator">?</span> <span class="token punctuation">(</span>image<span class="token punctuation">.</span>width <span class="token operator">*</span> stickerImage<span class="token punctuation">.</span>height<span class="token punctuation">)</span> <span class="token operator">/</span> stickerImage<span class="token punctuation">.</span>width <span class="token operator">:</span> <span class="token number">0</span><span class="token punctuation">;</span> <span class="token keyword">return</span> <span class="token punctuation">(</span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">KonvaImage</span></span> <span class="token attr-name">draggable</span> <span class="token attr-name">width</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>image<span class="token punctuation">.</span>width<span class="token punctuation">}</span></span> <span class="token attr-name">height</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>stickerHeight<span class="token punctuation">}</span></span> <span class="token attr-name">image</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>stickerImage<span class="token punctuation">}</span></span> <span class="token attr-name">x</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>image<span class="token punctuation">.</span>x<span class="token punctuation">}</span></span> <span class="token attr-name">y</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>image<span class="token punctuation">.</span>x<span class="token punctuation">}</span></span> <span class="token punctuation">/></span></span> <span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">;</span></code></pre></div> <p> </p> <ul> <li>Again, we load our image using the <code>useImage</code> hook</li> <li>To get the image to scale properly we add the <code>stickerHeight</code> calculation</li> <li>In the KonvaImage we add the <code>x</code> and <code>y</code> properties to determine where the sticker appears on the canvas when the button is clicked.</li> <li>The <code>draggable</code> property allows us to drag the image within the canvas!</li> </ul> <p>In App.js let's map over the images within our layer.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token punctuation">{</span>images<span class="token punctuation">.</span><span class="token function">map</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token parameter">image<span class="token punctuation">,</span> i</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">return</span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">IndividualSticker</span></span> <span class="token attr-name">key</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>i<span class="token punctuation">}</span></span> <span class="token attr-name">image</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>image<span class="token punctuation">}</span></span> <span class="token punctuation">/></span></span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">}</span></code></pre></div> <p><strong>Note:</strong> The sticker will always appear on top of the image if the stickers are below the image in our file (so there's no need for a z-index).</p> <p align="center"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <a class="gatsby-resp-image-link" href="/static/f6a08db25324b815113afa696fe82026/50383/stickers-image.png" style="display: block" target="_blank" rel="noopener"> <span class="gatsby-resp-image-background-image" style="padding-bottom: 92.97297297297298%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAATCAYAAACQjC21AAAACXBIWXMAABYlAAAWJQFJUiTwAAAEA0lEQVQ4y62U3VNaVxTF+WP6H/S5D53pQ6d9b9o8NG3Th047ndS0mSRNUm3UahrjB4ogqCgKiBIEBCMgYlRM8yVqNSEkRI2iIHgxgqgd9dcNmUynfehMMz0z6547++6zZu2197kqZWqKjNfLdnCU3O0Qu5MT7IWn2J8Ol8DDBxCZKe0H09Ol2F54ksLEbfLjIXbkXNbnIzPkJulyodp0u9k0Gol3d5IZ6CPvGmTX7aIgCccBH/6aSr758D1uVVaA30dB8nfdTvKDDl4O9JO1Wkmbuknq21hra0OVGR4mY+5l0WQk4xig4B0SeDgMBli1mbn4yfvozp3k03ffJikEhwE/u/J9T/JWrL1s9feh9FnY7O4iKVApvhHSFjO+ljoyg3b2hj0lQibGmaivQVt2QkrS0il7RNMEY2PsetwcCBa7O0jbLGSFMC1kqSLhTmCUNVGnv3KGZXM3hyO3yEsy4u3g5R+oP/0BmzfrMZZ9ROjXShCf8x4n+x4XARGxJme2bVJ2t5GUlK7aGhnh2tlT1Fd+xbULX5J13iyVTTBI8+mP0X9/kmjLj3z+zlv0n/8WQqPsDzlRHDaM5WX8rm+m0N8vfehks0hY9DCibSDWZ2ChR8e208G+EO76bhFqq+XxgJqJmvMEdbWE5fv+5BgFl4PmK19z7acvqD73GctGPcprD7e8w2RNvezYxL9B6Z4Q5sTLrbtT5ObGiDraiTk62Xt8nz+Wn5CZCZN1O3hq1BLv1fHIoGa9s51Ue7vsHaJwyMNGh4yMxSLdspJ32MmI0c/8XtJTdpKTbnLzYXKLv7G9OEN27i6KEObMZrZNPWSLzegwkNBpWTUYZA6dLjba9KR7e9iyWtix95OScbhnE5NHO0iO28lF58nH5tl6OIUyO03aZSddVGUUVQYdiTYdqxoNK7KrklLmWmsrSZGbNnWxIyOw1mvCXPUdG141cVe7kNwjH51jyWdnyVbDUkeDiJBB1mpYbdXwoqWF542NxIVUtW6/yUpzC+viQarLiCJKvXUVNEjHw4ZLRPQVrAespAI9JJxqntuqiGlqWGvRsKxWs9TYRLyhkVhdHbGmJlRLXSaiVdXEb9wgIQlz12uoPXMCg4xR1dmTRDqvE7M1MSvE0+oLPKg9x5Pqcp5W/0L0aiWPfr7KQnkFc5cuMy/vKiUyS2osREouuhIOo8gP4OWdOygz91GWn5FPvKCwvsrOSpz1hVk2ZLDTcls2QyGSwTE2RoOCURJ+PwmZXRWyDo4OSaY2+D9WiTCbSRNbmCsFjg4POT46eoXjIo7/wuv4v6BEGF2IMO53vyI8+gfJf4Sq+PBY9fSoK8nnXpZIi7E3LvluZJrqhjLqtRexeyx/U/km60850/dDeaHhvAAAAABJRU5ErkJggg==&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Woman in a pink dress with eiffel tower stickers" title="Woman in a pink dress with eiffel tower stickers" src="/static/f6a08db25324b815113afa696fe82026/50383/stickers-image.png" srcset="/static/f6a08db25324b815113afa696fe82026/1d79a/stickers-image.png 185w, /static/f6a08db25324b815113afa696fe82026/1efb2/stickers-image.png 370w, /static/f6a08db25324b815113afa696fe82026/50383/stickers-image.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </a> </span> </p> <h4>Multiple Stickers</h4> <p>Let's add multiple different styles of stickers 🤩</p> <p>Create a stickers data file to include an array of your chosen stickers. Include the url, width and alt name. Add as many as you like!</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">export</span> <span class="token keyword">const</span> stickersData <span class="token operator">=</span> <span class="token punctuation">[</span> <span class="token punctuation">{</span> url<span class="token operator">:</span> <span class="token string">"sticker-1.png"</span><span class="token punctuation">,</span> width<span class="token operator">:</span> <span class="token number">100</span><span class="token punctuation">,</span> alt<span class="token operator">:</span> <span class="token string">"Eiffel Tower"</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token punctuation">{</span> url<span class="token operator">:</span> <span class="token string">"sticker-2.png"</span><span class="token punctuation">,</span> width<span class="token operator">:</span> <span class="token number">150</span><span class="token punctuation">,</span> alt<span class="token operator">:</span> <span class="token string">"Statue of Liberty"</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token punctuation">{</span> url<span class="token operator">:</span> <span class="token string">"sticker-3.png"</span><span class="token punctuation">,</span> width<span class="token operator">:</span> <span class="token number">60</span><span class="token punctuation">,</span> alt<span class="token operator">:</span> <span class="token string">"Big Ben"</span> <span class="token punctuation">}</span> <span class="token punctuation">]</span><span class="token punctuation">;</span></code></pre></div> <p>In App.js import the stickers data and map over them to add all the buttons!</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token punctuation">{</span>stickersData<span class="token punctuation">.</span><span class="token function">map</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token parameter">sticker</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">return</span> <span class="token punctuation">(</span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>button</span> <span class="token attr-name">className</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>button<span class="token punctuation">"</span></span> <span class="token attr-name">onMouseDown</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token function">addStickerToPanel</span><span class="token punctuation">(</span><span class="token punctuation">{</span> src<span class="token operator">:</span> sticker<span class="token punctuation">.</span>url<span class="token punctuation">,</span> width<span class="token operator">:</span> sticker<span class="token punctuation">.</span>width<span class="token punctuation">,</span> x<span class="token operator">:</span> <span class="token number">100</span><span class="token punctuation">,</span> y<span class="token operator">:</span> <span class="token number">100</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">}</span></span> <span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>img</span> <span class="token attr-name">alt</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>sticker<span class="token punctuation">.</span>alt<span class="token punctuation">}</span></span> <span class="token attr-name">src</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>sticker<span class="token punctuation">.</span>url<span class="token punctuation">}</span></span> <span class="token attr-name">width</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>sticker<span class="token punctuation">.</span>width<span class="token punctuation">}</span></span> <span class="token punctuation">/></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>button</span><span class="token punctuation">></span></span> <span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">}</span></code></pre></div> <p align="center"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 613px; "> <a class="gatsby-resp-image-link" href="/static/ad604ec88ec49c081e62d941dcb33a34/5754a/multiple-stickers.png" style="display: block" target="_blank" rel="noopener"> <span class="gatsby-resp-image-background-image" style="padding-bottom: 105.40540540540542%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAVCAYAAABG1c6oAAAACXBIWXMAAAsSAAALEgHS3X78AAAEm0lEQVQ4y42T+0+TVxjH+9/sh/20H5f9umWJzl2yuGUzLnPJ2GJkixleZqKCRGWiXKRcRoUCLTex8FJasEXQWrTScqetvVDaQkvpHQqUbYCfHV4XlrhEd5Lzvnku5/t8z/d5jiJus7EybCZptZC2PSE7bmPd8YzcxDi5STs7zln2XPP8MTvF+r5P7DWRkxW56bHHJC2PiI+OsmTQE3syhmJZ349X3YRD3cyq1EtioJ/UoJ70/QFyD+6j/rWIws8OY1fVsTVqkmNJQz9xqY94nw5/eyvOpt/xN9QRFLYiOjRIsEOLVlnPQq+OjGmQlMnItvUhTxuVXPj2CNcLj3Lq0w/JW0bJDA+RFjkJwSg3ZKS/8hrqkrPE2jUsGQdQxEz3CXd30FRRSUhUzZqHSJmN/PXUSsflM7SXniRkqKWm8GsW73Wz8XBYsDcIQIk1Yz8V53+k6PujhFvURAYNKFbNJhY6NTSUXyLQe5esaYikycCubYzqUydQFR1n4W4l5T98jlOrFiwfkBoaENL0yQzPffcRxz5+l5AAjApbZrjQpcHQfJtAXw8Z4UybB1kTLC8eO0LfjV8YvvoTR997i/GGarYFw6Rgl9D3kRF/bclpas8UyAxlwH0Nw53txISgcdGQpNAhKzSKCJbWjhqm2qsYqS5ltK0Ot0ki+3iEpACL9+tY0XUT1/UQ6+km0HyH6KAAjBgNhLQaovfuEpN0cuWE6GJ6fpLEhIk5qY0FUzfboefkAi5i9seiuAARekaF9mHRjKCmFX9jgzw6iuUBPYHWFjzaNgEqKkr35LGI2a34za3EJy1knM9Y886QdE6QnBkn0tNFREyGr7UZf7OKYEsTz+uUhPQSiiVJwqNqZKKpUU7cv8aaUY9JXceA6gpJ+zBZn5OMZ5ao/RGrjocEO7Usa1qYVjXQX1WFX5x1KatZFFOiCPX1MlevxFB1g5CgH+5qJy30/K2ogMunv8JnFLZ7mpTTgVsSTDrLcKuUBAWIQ1lD/bVy3PW1zFTewi/0VITu6ZisrUJVXESgTc1yRzvRrg4Kj73PF4fewd5cRtDUhl+qw9dbyazmOvN1FfLLsFTepPFqMc7bt3GUleHp7kIxJ9pdfPJLSs99Q83ZAhbvNBIWT7Gk4BN+PnEYt16LS7rDs9abjNQUYyu7yHT5debKbmAuK6X51iVs165ivXCBWfF8FVHPc2xPRnC77Ew5xkh7Paz5fKSCAbKJGPlshj9z62yvZ8muRMh4vWT3t8dLwudlOeAl5fGIhjnJhkJibFZWSCRSRKMxVmNxAuEwAREIL0cILy3hdLnY2Nzi/y7F/mdnZ4ft/L+H9nZ32RW+V9eLvb3/7L1XbBlwc2ODSDh0cHArn2dm3sWuAH4ZzxEOLrwEffHizQxXIkv43PMHB/YZd0riPa+vyz77uJXqipJ/4nuvBZUBp2wWLCb9gTMvrq8RM7m1tSnbndp6Thw/JPz5N7JUxFajNFRfQll+nsWgT3amk3EaKy6TEl3eXw3KKxz+4G3y29tvBvT43QyN6jBbJKbnJ15qKDSbc1jZzK3J9vTkGJKu5UDT1wH+DQfRfzy/XNXUAAAAAElFTkSuQmCC&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Woman in a pink dress with eiffel tower, big ben and statue of liberty stickers" title="Woman in a pink dress with eiffel tower, big ben and statue of liberty stickers" src="/static/ad604ec88ec49c081e62d941dcb33a34/5754a/multiple-stickers.png" srcset="/static/ad604ec88ec49c081e62d941dcb33a34/1d79a/multiple-stickers.png 185w, /static/ad604ec88ec49c081e62d941dcb33a34/1efb2/multiple-stickers.png 370w, /static/ad604ec88ec49c081e62d941dcb33a34/5754a/multiple-stickers.png 613w" sizes="(max-width: 613px) 100vw, 613px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </a> </span> </p> <p><strong>Checkpoint: Draggable stickers that work well on mobile and desktop! 🎉</strong></p> <p>⁉️ WAIT, but I want to be able to delete the stickers!?</p> <p>✅ Okay, let's continue with the tutorial. Things get a little more complicated here!</p> <h3>Deleting the stickers</h3> <p>In order to delete stickers from the canvas we add a cross icon to the top right corner of each sticker.</p> <ul> <li>On mobile the cross appears on long tap of the sticker</li> <li>On desktop the cross appears when you hover over the sticker</li> </ul> <p>Let's add another <code>KonvaImage</code> to our individual sticker and wrap it in a <code>Group</code> so the sticker and cross icon are grouped.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">export</span> <span class="token keyword">const</span> <span class="token function-variable function">IndividualSticker</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token parameter"><span class="token punctuation">{</span> image<span class="token punctuation">,</span> onDelete<span class="token punctuation">,</span> onDragEnd <span class="token punctuation">}</span></span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">const</span> <span class="token punctuation">[</span>stickerImage<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token function">useImage</span><span class="token punctuation">(</span>image<span class="token punctuation">.</span>src<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">const</span> <span class="token punctuation">[</span>deleteImage<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token function">useImage</span><span class="token punctuation">(</span><span class="token string">"cancel.svg"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">const</span> stickerWidth <span class="token operator">=</span> image<span class="token punctuation">.</span>width<span class="token punctuation">;</span> <span class="token keyword">const</span> stickerHeight <span class="token operator">=</span> stickerImage <span class="token operator">?</span> <span class="token punctuation">(</span>image<span class="token punctuation">.</span>width <span class="token operator">*</span> stickerImage<span class="token punctuation">.</span>height<span class="token punctuation">)</span> <span class="token operator">/</span> stickerImage<span class="token punctuation">.</span>width <span class="token operator">:</span> <span class="token number">0</span><span class="token punctuation">;</span> <span class="token keyword">return</span> <span class="token punctuation">(</span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">Group</span></span> <span class="token attr-name">draggable</span> <span class="token attr-name">x</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>image<span class="token punctuation">.</span>x<span class="token punctuation">}</span></span> <span class="token attr-name">y</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>image<span class="token punctuation">.</span>y<span class="token punctuation">}</span></span> <span class="token attr-name">onDragEnd</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span><span class="token punctuation">(</span><span class="token parameter">event</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token function">onDragEnd</span><span class="token punctuation">(</span>event<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">}</span></span> <span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">KonvaImage</span></span> <span class="token attr-name">width</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>image<span class="token punctuation">.</span>width<span class="token punctuation">}</span></span> <span class="token attr-name">height</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>stickerHeight<span class="token punctuation">}</span></span> <span class="token attr-name">image</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>stickerImage<span class="token punctuation">}</span></span> <span class="token punctuation">/></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">KonvaImage</span></span> <span class="token attr-name">onTouchStart</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>onDelete<span class="token punctuation">}</span></span> <span class="token attr-name">onClick</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>onDelete<span class="token punctuation">}</span></span> <span class="token attr-name">image</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>deleteImage<span class="token punctuation">}</span></span> <span class="token attr-name">width</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span><span class="token number">25</span><span class="token punctuation">}</span></span> <span class="token attr-name">height</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span><span class="token number">25</span><span class="token punctuation">}</span></span> <span class="token attr-name">offsetX</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span><span class="token operator">-</span>stickerWidth <span class="token operator">/</span> <span class="token number">2</span> <span class="token operator">-</span> <span class="token number">20</span><span class="token punctuation">}</span></span> <span class="token punctuation">/></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span><span class="token class-name">Group</span></span><span class="token punctuation">></span></span> <span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">;</span></code></pre></div> <ul> <li>Render a cross icon using the <code>useImage</code> hook</li> <li>Add the <code>draggable</code>, <code>x</code> and <code>y</code> properties to the <code>Group</code> so that they are common to both images</li> <li>Use the offset properties on the delete button to position the cross where you like</li> <li>Add the <code>onClick</code> property and pass the <code>onDelete</code> function. Also add the <code>onTouchStart</code> property so that this works on mobile too!</li> <li> <p>Pass an <code>onDelete</code> function detailed in App.js</p> <ul> <li>Use a splice to remove the image from the local state when the delete button is tapped/clicked</li> </ul> </li> </ul> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx">onDelete<span class="token operator">=</span><span class="token punctuation">{</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">const</span> newImages <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token operator">...</span>images<span class="token punctuation">]</span><span class="token punctuation">;</span> images<span class="token punctuation">.</span><span class="token function">splice</span><span class="token punctuation">(</span>i<span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">setImages</span><span class="token punctuation">(</span>newImages<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">}</span></code></pre></div> <p>We need to ensure that the <code>x</code> and <code>y</code> position of each sticker in the local state is updated appropriately - we make use of the Group's <code>onDragEnd</code> property. We pass in the following function from the App.js file. This updates the <code>x</code> and <code>y</code> position of the sticker at the end of the drag movement.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx">onDragEnd<span class="token operator">=</span><span class="token punctuation">{</span><span class="token punctuation">(</span><span class="token parameter">event</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> image<span class="token punctuation">.</span>x <span class="token operator">=</span> event<span class="token punctuation">.</span>target<span class="token punctuation">.</span><span class="token function">x</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> image<span class="token punctuation">.</span>y <span class="token operator">=</span> event<span class="token punctuation">.</span>target<span class="token punctuation">.</span><span class="token function">y</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">}</span></code></pre></div> <p> <br> <strong>Checkpoint:</strong> The cross shows when we add a sticker to the photo. When we tap/click the icon, it deletes the sticker as expected</p> <p align="center"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 608px; "> <a class="gatsby-resp-image-link" href="/static/d4c3796db7507768ef8f067d7163aeea/18872/stickers-with-crosses.png" style="display: block" target="_blank" rel="noopener"> <span class="gatsby-resp-image-background-image" style="padding-bottom: 67.56756756756756%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAAAsSAAALEgHS3X78AAADiklEQVQ4y02T208bdBTH++f44INPvs+ZaBZ98UGT6bJkYmZM5hKQZAYRJ2zIuLS0hRa6trSFlpbeRsulDCi3DqcyoYw7Q+6F0hZ63RgxHw8diXs4yS+/3/l9ft/zPeenSPh9nPT3kXkcIjc6TCE8wsvxcDH4PQJPn0hMczoxxivZOz/PjzwmNxQiM9BPOhgk4fUSczjYtlhQxK1WUi4naZ+HbK+fQl8v+b4AyKVJ5QO++fgSptJb/Ds0yMv+IIWgnEtexuvhWO7tmIwcmk3st+nZ1KhRJBx2TrxuMo985ARWGAhyNjzEkd9NxdUrWCpvcO3Se8zoNDA2Sr4/wKmAY+4e5kwmVtp0HNosRehuWxuKZLeDtLyWDYi6cwUCZCLMnEFLXcmnpEI67He+YuS3KtkfK6p/LdB9VzdWlZak00Gqy/ZGZXs7ipSz+61yA2+Ak+NMNNZQffWy+NOM88dreCtLi/t5qeK1xJ7LgbmxibSAkxfAmMFwodDnJSsl54MX/o2FsX5fQs2NK2wZf+X2R++i/PoLCEszAj7OJG+ty4y+roJEdxepzreACbt46HEL0F8EnjflVDwcVFax6NEwVVvOUPNdelXVFCaGyQvwlTy+0W0hYGjgwG4jZbP+Dzyy2Th2OjkRk3N+LwW/h/QfEXILEVZ9BubtOjJ/T3K2uUIy+pSTUJB8j4uUeBfvtJKwWoibzUXYrl6P4kBmZ830kGiLhqx0O+ftKQKPIj72w27Ss1Nk5sbJLM2SXviLWP8jFppVHItvCYuZQ7kbM7QXYVstLSh2hDxv0JOwd5F2u8hKpMJDHIxaOAg7ya/Ok1uNkl98xnI4yLSxpago3mHiwGhg39DGnszgtsD+0WhQrMliWPVAWt9JytFFxuVio0PLTkDJVtBI4tk02ZUo6ZkptJU3aSor4cTSIRAde3odu62tbGm1bKjVrKtUKJa0aixVZSQlKSnmHkuTnrfWsu6qZdZYzd6Ih9STAImQmarvPuHm5x+y16JlWwCbqmY2lCrWm5pYbWhgub4eRWPZderuXMdRVUqyo4O4TH9Uc58JVQURzU+8cKtZtFSzZPyZW1++z2cfvMO2QF40NLJW38BKXR1L92tZuHePaE0NCuMvtwm03qWzupx9AR7K347JH10PDxL7c4r4TISdyRDLA14ctT9gK/+WDW0ra2oNK1LislLJogz4c1E4Jwr/A1IKaP7Yc0d7AAAAAElFTkSuQmCC&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Woman in a pink dress with stickers that have crosses in the top right corner" title="Woman in a pink dress with stickers that have crosses in the top right corner" src="/static/d4c3796db7507768ef8f067d7163aeea/18872/stickers-with-crosses.png" srcset="/static/d4c3796db7507768ef8f067d7163aeea/1d79a/stickers-with-crosses.png 185w, /static/d4c3796db7507768ef8f067d7163aeea/1efb2/stickers-with-crosses.png 370w, /static/d4c3796db7507768ef8f067d7163aeea/18872/stickers-with-crosses.png 608w" sizes="(max-width: 608px) 100vw, 608px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </a> </span> </p> <p>Now, we only want to show the cross when we hover on desktop or long press on mobile - let's continue!</p> <p>We use the <a href="https://github.com/streamich/react-use" target="_blank">react-use</a> <code>useLongPress</code> and <code>useHoverDirty</code> to show the cross on long tap (mobile) or on hover (desktop).</p> <p>Let's amend IndividualSticker.tsx to look like the following...</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">const</span> imageRef <span class="token operator">=</span> <span class="token function">useRef</span><span class="token punctuation">(</span><span class="token keyword">null</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">const</span> isHovered <span class="token operator">=</span> <span class="token function">useHoverDirty</span><span class="token punctuation">(</span>imageRef<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">const</span> <span class="token punctuation">[</span>showDeleteButton<span class="token punctuation">,</span> setShowDeleteButton<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token function">useState</span><span class="token punctuation">(</span><span class="token boolean">false</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">const</span> <span class="token punctuation">[</span>isDragging<span class="token punctuation">,</span> setIsDragging<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token function">useState</span><span class="token punctuation">(</span><span class="token boolean">false</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">const</span> <span class="token function-variable function">onLongPress</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token function">setShowDeleteButton</span><span class="token punctuation">(</span><span class="token boolean">true</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">;</span> <span class="token keyword">const</span> longPressEvent <span class="token operator">=</span> <span class="token function">useLongPress</span><span class="token punctuation">(</span>onLongPress<span class="token punctuation">,</span> <span class="token punctuation">{</span> delay<span class="token operator">:</span> <span class="token number">200</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">useEffect</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>isHovered<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token function">setShowDeleteButton</span><span class="token punctuation">(</span><span class="token boolean">true</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token punctuation">{</span> <span class="token function">setTimeout</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token function">setShowDeleteButton</span><span class="token punctuation">(</span><span class="token boolean">false</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token number">2000</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token punctuation">[</span>isHovered<span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">return</span> <span class="token punctuation">(</span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">Group</span></span> <span class="token attr-name">draggable</span> <span class="token attr-name">x</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>image<span class="token punctuation">.</span>x<span class="token punctuation">}</span></span> <span class="token attr-name">y</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>image<span class="token punctuation">.</span>y<span class="token punctuation">}</span></span> <span class="token attr-name">onDragStart</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token function">setIsDragging</span><span class="token punctuation">(</span><span class="token boolean">true</span><span class="token punctuation">)</span><span class="token punctuation">}</span></span> <span class="token attr-name">onDragEnd</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span><span class="token punctuation">(</span><span class="token parameter">event</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token function">setIsDragging</span><span class="token punctuation">(</span><span class="token boolean">false</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">onDragEnd</span><span class="token punctuation">(</span>event<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">}</span></span> <span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">KonvaImage</span></span> <span class="token attr-name">ref</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>imageRef<span class="token punctuation">}</span></span> <span class="token attr-name">width</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>image<span class="token punctuation">.</span>width<span class="token punctuation">}</span></span> <span class="token attr-name">height</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>stickerHeight<span class="token punctuation">}</span></span> <span class="token attr-name">image</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>stickerImage<span class="token punctuation">}</span></span> <span class="token spread"><span class="token punctuation">{</span><span class="token punctuation">...</span><span class="token attr-value">longPressEvent</span><span class="token punctuation">}</span></span> <span class="token punctuation">/></span></span><span class="token plain-text"> </span><span class="token punctuation">{</span>showDeleteButton <span class="token operator">&amp;&amp;</span> <span class="token operator">!</span>isDragging <span class="token operator">&amp;&amp;</span> <span class="token punctuation">(</span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">KonvaImage</span></span> <span class="token attr-name">onTouchStart</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>onDelete<span class="token punctuation">}</span></span> <span class="token attr-name">onClick</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>onDelete<span class="token punctuation">}</span></span> <span class="token attr-name">image</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>deleteImage<span class="token punctuation">}</span></span> <span class="token attr-name">width</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span><span class="token number">25</span><span class="token punctuation">}</span></span> <span class="token attr-name">height</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span><span class="token number">25</span><span class="token punctuation">}</span></span> <span class="token attr-name">offsetX</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span><span class="token operator">-</span>stickerWidth <span class="token operator">/</span> <span class="token number">2</span> <span class="token operator">-</span> <span class="token number">20</span><span class="token punctuation">}</span></span> <span class="token punctuation">/></span></span> <span class="token punctuation">)</span><span class="token punctuation">}</span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span><span class="token class-name">Group</span></span><span class="token punctuation">></span></span> <span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre></div> <ul> <li>We add a <code>ref</code> to the image that will be hovered. <code>useHoverDirty</code> is true when the image is hovered</li> <li>We store the <code>isDragging</code> and <code>showDeleteButton</code> booleans in the local state using the <code>useState</code> hook</li> <li>When we long tap and image we set the <code>showDeleteButton</code> state to true after a delay of 200ms</li> <li>The <code>useEffect</code> ensures that the delete button disappears after the 2s of the image being hovered if it is not deleted.</li> <li>We only display the delete button if the the <code>showDelete</code> button state is true and <code>isDragging</code> is false. This ensures that we don't show the delete button when we're dragging. This is a little hacky but it considers all the cases for mobile and desktop use!</li> </ul> <p>We now need to ensure that the cross disappears if a user taps an area not associated with the sticker.</p> <p>In App.js</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">const</span> <span class="token function-variable function">addStickerToPanel</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token parameter"><span class="token punctuation">{</span> src<span class="token punctuation">,</span> width<span class="token punctuation">,</span> x<span class="token punctuation">,</span> y <span class="token punctuation">}</span></span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token function">setImages</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token parameter">currentImages</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">[</span> <span class="token operator">...</span>currentImages<span class="token punctuation">,</span> <span class="token punctuation">{</span> width<span class="token punctuation">,</span> x<span class="token punctuation">,</span> y<span class="token punctuation">,</span> src<span class="token punctuation">,</span> resetButtonRef<span class="token operator">:</span> <span class="token function">createRef</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">}</span> <span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">;</span> <span class="token keyword">const</span> resetAllButtons <span class="token operator">=</span> <span class="token function">useCallback</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> images<span class="token punctuation">.</span><span class="token function">forEach</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token parameter">image</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>image<span class="token punctuation">.</span>resetButtonRef<span class="token punctuation">.</span>current<span class="token punctuation">)</span> <span class="token punctuation">{</span> image<span class="token punctuation">.</span>resetButtonRef<span class="token punctuation">.</span><span class="token function">current</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token punctuation">[</span>images<span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">const</span> handleCanvasClick <span class="token operator">=</span> <span class="token function">useCallback</span><span class="token punctuation">(</span> <span class="token punctuation">(</span><span class="token parameter">event</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>event<span class="token punctuation">.</span>target<span class="token punctuation">.</span>attrs<span class="token punctuation">.</span>id <span class="token operator">===</span> <span class="token string">"backgroundImage"</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token function">resetAllButtons</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token punctuation">[</span>resetAllButtons<span class="token punctuation">]</span> <span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre></div> <ul> <li>We create a new <code>ref</code> on each button using React's <code>createRef</code></li> <li>We add the <code>handleCanvasClick</code> which resets all the refs on the buttons so that we don't display the cross!</li> </ul> <p>In the <code>Stage</code> add the <code>onClick</code> and <code>onTap</code> to handle this.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">Stage</span></span> <span class="token attr-name">width</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span><span class="token number">600</span><span class="token punctuation">}</span></span> <span class="token attr-name">height</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span><span class="token number">400</span><span class="token punctuation">}</span></span> <span class="token attr-name">onClick</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>handleCanvasClick<span class="token punctuation">}</span></span> <span class="token attr-name">onTap</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>handleCanvasClick<span class="token punctuation">}</span></span> <span class="token punctuation">></span></span></code></pre></div> <p> </p> <h3>The Result</h3> <p align="center"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 609px; "> <a class="gatsby-resp-image-link" href="/static/df6e551890b40133e5fc154d25890388/d0d8c/finished-stickers.png" style="display: block" target="_blank" rel="noopener"> <span class="gatsby-resp-image-background-image" style="padding-bottom: 105.94594594594595%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAVCAYAAABG1c6oAAAACXBIWXMAAAsSAAALEgHS3X78AAAEd0lEQVQ4y42S+0+TZxTH+/9sf8M02SXZsmXZkmUx08UlM9ucw/2wHya7qUOmKEjHrUBLKaXFAi2ltAUKiICIF8pQWuj9LS29UAqlXCqK8NnTV8UflrG9yXmfnHOe83nO+T6PIuawE7dZSQ06yYy4WB0dITc+Sm5ilI3JMQr3JtmampDXYqyYK+7JjgyRcQ2QctpZsvUSMbYTG+xHIXWaiAonYekkbethxdFHdsDOmsspW+W3p/j8nSOM1VVTuDUsciLv7CMjIGmrmVR3JzFRH2hsINpjRrFoMRPrvEGq18Ky3SYKHGQHHTy5PcbA9T+4+NXHqM6d5MyHb7M9NsLakJNVkS8enLZZ8Om1RDsMRLQa4n02FPHeHpa6TXJ3GbFpVXS16nKwO3Ub3U/fY7p0BsleT93Zz1js6WJzVMgiJsjae/EYWnnU0kz0hgFJAJfsRaD1JdD6YlyHDHwq9Ks4dQz1DyeY11/h4hcf4OvQsXVriGy/TQbebW0iLQ5ZEhMWO1wS9QfAhNVCXNiaAOaGB1jus/DL8ffpV5biuPANx468zrS6lsJNFyt9Vhlovl5O0NhGUgaqSYgLVsSEhnFTh7iULlyaFmJCgryrn9iQg2mrGnd7NeMNFdw16/DfFN2PD7MmYHMGDXW/ljBeX0VKXGxI0/x85GhXF5KhnYy5C52yjkdGI7l+oZHHzfJ9J54+I5HhTgrSAvngHOnp2yTMJipLv6a67DRXzp1iQegYampkUTSjkEwmJL2euGhbc7Ucv8nAWvFZzNwhOtLO8swEOe89GZaanSLlviNrJrXrhGkJapsJCqCvvhZJSKYIG4tXriVs0KNRXuRhh5YVcVJ4oBPJpSY+6SQX8IgOfQwam7lpVJESb1bStRBp1RAWsICmCW+tkrClG0WwXU9A3URIp+Veq4pgRxsJ0w3+aqkgZK/F09tG1utmw/uAit9KuFTyKWFtA4EmFT5VPQsNdXiFzVZVEhBaKhZatcyLRLBFTUzfRlw80rC+lcnGMqbU57mjvoQ02IZkq6fk5Jt8+clRAatlXozoqa1hTqnkobDpy5dZEPor5lQqQa8SSaW8KSS6DTc3MqOu4Za2hgWHEW+vFrfhT747cZSzH72Bt+oabgF4UF7O/bIy7pb9znhpKbPqZhRpn4+U10vG72dFWMbvI+NbIBePk8+ukMss8zi/zpPNDZKLIRI+LxuRCOvCcuEwuVBIrCFWAwHyokYRTyZJpNMkMxkWEwniqRSSSKSEH43FCIvC7UKB//spir/dp0/ZefyqaH9vj71nz/6xea8YF7Z/iMnAtdUsmXTyoHBzcwv37CP29/dlP7+eIxL2Hxz2Mv6vHUYjQaLh4EFwu/CYDquDwotRJyeGqa68cNDlfwInR5xMjQ4eBPP5PG16HTs7O7Kv01Rz8vh77O7uPu/yMGBEClBz9Ufqq35mKRmTg2mxNl8/z8b6muwrr5Xy7luvvdL4MOCsZ4ahcRuusV7m/Z4Xmq0y555ke2tD9qfvj2Hp1r7EHQr8GyVbgvcqntl6AAAAAElFTkSuQmCC&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Woman in a pink dress with stickers" title="Woman in a pink dress with stickers" src="/static/df6e551890b40133e5fc154d25890388/d0d8c/finished-stickers.png" srcset="/static/df6e551890b40133e5fc154d25890388/1d79a/finished-stickers.png 185w, /static/df6e551890b40133e5fc154d25890388/1efb2/finished-stickers.png 370w, /static/df6e551890b40133e5fc154d25890388/d0d8c/finished-stickers.png 609w" sizes="(max-width: 609px) 100vw, 609px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </a> </span> </p> <p>Using react-konva to build a great photobooth experience within a react app was really fun and I'll look forward to using this framework again! Check out more react-konva examples <a href="https://codesandbox.io/examples/package/react-konva" target="_blank">here</a> and don't forget to checkout my <a href="https://codesandbox.io/s/elated-shadow-ed4rj" target="_blank">codesandbox!</a></p> <p><em>Sources + links:</em></p> <ol> <li><a href="https://konvajs.org/">https://konvajs.org/</a></li> <li><a href="https://github.com/konvajs/react-konva">https://github.com/konvajs/react-konva</a></li> <li><a href="https://github.com/streamich/react-use">https://github.com/streamich/react-use</a></li> </ol><![CDATA[Create resizeable split panels in React]]>/2020/11/react-resizeable-split-panels//2020/11/react-resizeable-split-panels/Mon, 09 Nov 2020 00:00:00 GMT<p>Resizable containers empower different users to customize a UI to emphasise what they find most important. A basic implementation of this is a split view or split pane, allowing users to enlarge content that is more relevant to them.</p> <p align="center"> <img alt="Resizeable split panes in action" src="/1f9ddf8a4877dc953614f038aa0df6a8/split-pane-demo.gif" style="margin-top:30px;margin-bottom:20px;width:60%;max-width:60%"> </p> <p>There are libraries that can implement this feature for you, but creating it yourself gives you full control over the styling and functionality, so you can rapidly tailor it to your project. The full feature can be done in under 150 lines, with no dependencies, and in my case it was faster to add to my project than using a third-party library. It's also a component that covers a wide range of the features React has to offer, which is great for quickly getting to grips with what the library can do.</p> <p>You can find the finished component right <a href="https://codesandbox.io/s/splitview-p37yw">here</a>.</p> <p>(Credit to <a href="https://tinloof.com/blog/how-to-make-your-own-splitpane-react-component-with-0-dependencies/">Seif Ghezala</a> for the initial inspiration I've expanded on in this article)</p> <h3>Getting started</h3> <h4>Creating the component</h4> <p>Let's start out by creating a functional component. We want to be able to pass in the content to be displayed in each panel, so let's go ahead and pass in the LeftContent and RightContent as props, and wrap them in styled div so they're displayed side by side.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">import</span> React <span class="token keyword">from</span> <span class="token string">"react"</span><span class="token punctuation">;</span> <span class="token keyword">interface</span> <span class="token class-name">SplitViewProps</span> <span class="token punctuation">{</span> left<span class="token operator">:</span> React<span class="token punctuation">.</span>ReactElement<span class="token punctuation">;</span> right<span class="token operator">:</span> React<span class="token punctuation">.</span>ReactElement<span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token keyword">export</span> <span class="token keyword">const</span> SplitView<span class="token operator">:</span> React<span class="token punctuation">.</span>FunctionComponent<span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">SplitViewProps</span></span><span class="token punctuation">></span></span><span class="token plain-text"> = ({ left, right, }) => </span><span class="token punctuation">{</span> <span class="token keyword">return</span> <span class="token punctuation">(</span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span> <span class="token attr-name">className</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>splitView<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token punctuation">{</span>left<span class="token punctuation">}</span><span class="token plain-text"> </span><span class="token punctuation">{</span>right<span class="token punctuation">}</span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>div</span><span class="token punctuation">></span></span> <span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token plain-text">;</span></code></pre></div> <div class="gatsby-highlight" data-language="css"><pre class="language-css"><code class="language-css"><span class="token selector">.splitView</span> <span class="token punctuation">{</span> <span class="token property">height</span><span class="token punctuation">:</span> 100%<span class="token punctuation">;</span> <span class="token property">display</span><span class="token punctuation">:</span> flex<span class="token punctuation">;</span> <span class="token property">flex-direction</span><span class="token punctuation">:</span> row<span class="token punctuation">;</span> <span class="token property">align-items</span><span class="token punctuation">:</span> flex-start<span class="token punctuation">;</span> <span class="token punctuation">}</span></code></pre></div> <p>Now let's add the divider. I'm going to keep things simple and just use a div with a margin. The width will be fixed at a few pixels, and I want the height to grow with the content in the panels either side, which we can do with height 100%.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"> <span class="token keyword">return</span> <span class="token punctuation">(</span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span> <span class="token attr-name">className</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>splitView<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token punctuation">{</span>left<span class="token punctuation">}</span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span> <span class="token attr-name">className</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>divider<span class="token punctuation">"</span></span><span class="token punctuation">/></span></span><span class="token plain-text"> </span><span class="token punctuation">{</span>right<span class="token punctuation">}</span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>div</span><span class="token punctuation">></span></span> <span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre></div> <div class="gatsby-highlight" data-language="css"><pre class="language-css"><code class="language-css"><span class="token selector">.divider</span> <span class="token punctuation">{</span> <span class="token property">width</span><span class="token punctuation">:</span> 2px<span class="token punctuation">;</span> <span class="token property">height</span><span class="token punctuation">:</span> 100%<span class="token punctuation">;</span> <span class="token property">margin</span><span class="token punctuation">:</span> 1rem<span class="token punctuation">;</span> <span class="token property">border</span><span class="token punctuation">:</span> 2px solid #808080<span class="token punctuation">;</span> <span class="token punctuation">}</span></code></pre></div> <p>Alright here's our basic component set up.</p> <p align="center"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 474px; "> <a class="gatsby-resp-image-link" href="/static/cd99a92a72e49760ba809b54521765ef/5595f/basic-component.png" style="display: block" target="_blank" rel="noopener"> <span class="gatsby-resp-image-background-image" style="padding-bottom: 17.837837837837835%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAECAYAAACOXx+WAAAACXBIWXMAABYlAAAWJQFJUiTwAAAAlUlEQVQY04WPvQqFIACFff9ncHZt6BGSQCEoU7AhRyEoR1OITuTQcu/tnuUs54eP4EXneX71N5F1XWGMgRAC8zyj6zpYa7HvewnknLFt21O4M23bQikFrXXJDsNQPIQAMk0TqqqClBLjOIIxBs45YoxlIKWEZVmeQecc6roGpbSc931f+k3TwHsP8g/hxjyO4wP5F/4FbH8tlhLthz8AAAAASUVORK5CYII=&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Base split view component" title="Base split view component" src="/static/cd99a92a72e49760ba809b54521765ef/5595f/basic-component.png" srcset="/static/cd99a92a72e49760ba809b54521765ef/1d79a/basic-component.png 185w, /static/cd99a92a72e49760ba809b54521765ef/1efb2/basic-component.png 370w, /static/cd99a92a72e49760ba809b54521765ef/5595f/basic-component.png 474w" sizes="(max-width: 474px) 100vw, 474px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </a> </span> </p> <p>Now for the fun part!</p> <h4>Time to start resizing</h4> <p>So the end goal is to click and drag the divider and resize both panels accordingly. The trick here is to resize just one of the panels, and then let flex-grow and flex-shrink take care of the other.</p> <p>Let's control the panel on the left. We're going to need a way of storing and setting its width so we can create some state for our component using the useState hook.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">const</span> <span class="token punctuation">[</span>leftWidth<span class="token punctuation">,</span> setLeftWidth<span class="token punctuation">]</span> <span class="token operator">=</span> useState<span class="token operator">&lt;</span><span class="token keyword">undefined</span> <span class="token operator">|</span> number<span class="token operator">></span><span class="token punctuation">(</span><span class="token keyword">undefined</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre></div> <p>The panel on the left is going to have quite a bit of its own functionality so lets move that div into its own new component called LeftPanel. We want it to use the state we just created so lets pass in leftWidth and the setter too.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">const</span> LeftPanel<span class="token operator">:</span> React<span class="token punctuation">.</span>FunctionComponent<span class="token operator">&lt;</span><span class="token punctuation">{</span> leftWidth<span class="token operator">:</span> number <span class="token operator">|</span> <span class="token keyword">undefined</span><span class="token punctuation">;</span> <span class="token function-variable function">setLeftWidth</span><span class="token operator">:</span> <span class="token punctuation">(</span><span class="token parameter">value<span class="token operator">:</span> number</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token keyword">void</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token operator">></span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token parameter"><span class="token punctuation">{</span> children<span class="token punctuation">,</span> leftWidth<span class="token punctuation">,</span> setLeftWidth <span class="token punctuation">}</span></span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">return</span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span><span class="token punctuation">></span></span><span class="token punctuation">{</span>children<span class="token punctuation">}</span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>div</span><span class="token punctuation">></span></span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">;</span></code></pre></div> <p>Time for the magic. The ref!</p> <p>React refs allow us to access the properties of elements in the DOM. If we use it on the div around our left panel content, we'll be able to both get and set the width of it directly.</p> <p>To use refs, first create one using React's <code>createRef</code> function. Then attach it to the desired element using the ref property.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"> <span class="token keyword">const</span> leftRef <span class="token operator">=</span> createRef<span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">HTMLDivElement</span></span><span class="token punctuation">></span></span><span class="token plain-text">(); return </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span> <span class="token attr-name">ref</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>leftRef<span class="token punctuation">}</span></span><span class="token punctuation">></span></span><span class="token punctuation">{</span>children<span class="token punctuation">}</span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>div</span><span class="token punctuation">></span></span><span class="token plain-text">;</span></code></pre></div> <p>By doing this, the properties of that element are now accessible under <code>leftRef.current</code>.</p> <p>Time to connect our state and the ref with another React hook, useEffect. First we want to make sure that if our state isn't defined it gets initialized to the width of the left panel.</p> <p>Then we want to make sure that whenever our state changes, we change the width of the left panel too. We can access the width using <code>current.clientWidth</code>.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">const</span> LeftPanel<span class="token operator">:</span> React<span class="token punctuation">.</span>FunctionComponent<span class="token operator">&lt;</span><span class="token punctuation">{</span> leftWidth<span class="token operator">:</span> number <span class="token operator">|</span> <span class="token keyword">undefined</span><span class="token punctuation">;</span> <span class="token function-variable function">setLeftWidth</span><span class="token operator">:</span> <span class="token punctuation">(</span><span class="token parameter">value<span class="token operator">:</span> number</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token keyword">void</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token operator">></span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token parameter"><span class="token punctuation">{</span> children<span class="token punctuation">,</span> leftWidth<span class="token punctuation">,</span> setLeftWidth <span class="token punctuation">}</span></span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">const</span> leftRef <span class="token operator">=</span> createRef<span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">HTMLDivElement</span></span><span class="token punctuation">></span></span><span class="token plain-text">(); React.useEffect(() => </span><span class="token punctuation">{</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>leftRef<span class="token punctuation">.</span>current<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token operator">!</span>leftWidth<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token function">setLeftWidth</span><span class="token punctuation">(</span>leftRef<span class="token punctuation">.</span>current<span class="token operator">?.</span>clientWidth<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">return</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> leftRef<span class="token punctuation">.</span>current<span class="token punctuation">.</span>style<span class="token punctuation">.</span>width <span class="token operator">=</span> <span class="token template-string"><span class="token template-punctuation string">`</span><span class="token interpolation"><span class="token interpolation-punctuation punctuation">${</span>leftWidth<span class="token interpolation-punctuation punctuation">}</span></span><span class="token string">px</span><span class="token template-punctuation string">`</span></span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span><span class="token plain-text">, [leftRef, leftWidth, setLeftWidth]); return </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span> <span class="token attr-name">ref</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>leftRef<span class="token punctuation">}</span></span><span class="token punctuation">></span></span><span class="token punctuation">{</span>children<span class="token punctuation">}</span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>div</span><span class="token punctuation">></span></span><span class="token plain-text">; };</span></code></pre></div> <p>Now whenever we change the leftWidth state in our original SplitView component, it will change the width of the left panel!</p> <p>Notice the array after the contents of the useEffect. That dependency array makes sure that the code in the effect runs whenever either of those values change.</p> <p>So we could do the same thing for the right panel, but as I said before, we can make things much simpler by giving it flex-grow (here I've used the shorthand <code>flex: 1</code>).</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">export</span> <span class="token keyword">const</span> SplitView<span class="token operator">:</span> React<span class="token punctuation">.</span>FunctionComponent<span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">SplitViewProps</span></span><span class="token punctuation">></span></span><span class="token plain-text"> = ({ left, right}) => </span><span class="token punctuation">{</span> <span class="token keyword">const</span> <span class="token punctuation">[</span>leftWidth<span class="token punctuation">,</span> setLeftWidth<span class="token punctuation">]</span> <span class="token operator">=</span> useState<span class="token operator">&lt;</span><span class="token keyword">undefined</span> <span class="token operator">|</span> number<span class="token operator">></span><span class="token punctuation">(</span><span class="token keyword">undefined</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">return</span> <span class="token punctuation">(</span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span> <span class="token attr-name">className</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>splitView<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">LeftPanel</span></span> <span class="token attr-name">leftWidth</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>leftWidth<span class="token punctuation">}</span></span> <span class="token attr-name">setLeftWidth</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>setLeftWidth<span class="token punctuation">}</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token punctuation">{</span>left<span class="token punctuation">}</span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span><span class="token class-name">LeftPanel</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span> <span class="token attr-name">className</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>divider<span class="token punctuation">"</span></span><span class="token punctuation">/></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span> <span class="token attr-name">className</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>rightPane<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token punctuation">{</span>right<span class="token punctuation">}</span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>div</span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>div</span><span class="token punctuation">></span></span> <span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token plain-text">;</span></code></pre></div> <div class="gatsby-highlight" data-language="css"><pre class="language-css"><code class="language-css"><span class="token selector">.rightPane</span> <span class="token punctuation">{</span> <span class="token property">flex</span><span class="token punctuation">:</span> 1<span class="token punctuation">;</span> <span class="token punctuation">}</span></code></pre></div> <p>Now wasn't that easier?</p> <p>We're all set to resize both of our panels. We just need something to trigger it.</p> <h4>Getting the mouse involved</h4> <p>We want to resize our panels when we click and drag the separator. We can break that down into 3 stages:</p> <ul> <li>Clicking down on the separator</li> <li>Dragging the mouse</li> <li>Releasing the separator</li> </ul> <p>Conveniently, we can capture each of these stages with 3 mouse events;</p> <ul> <li>onMouseDown: Set a flag to start tracking mouse movements. Store the initial position of the mouse at this moment.</li> <li>onMouseMove: If the flag is set, track the mouse's horizontal position, and resize the left panel accordingly</li> <li>onMouseUp: Clear the flag</li> </ul> <p>Let's start with onMouseDown. When the user clicks on the separator, we want to store the mouse's horizontal position at that moment, and set a flag to start tracking any horizontal movement after that. Sounds like we need some more useState hooks.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"> <span class="token keyword">const</span> <span class="token punctuation">[</span>separatorXPosition<span class="token punctuation">,</span> setSeparatorXPosition<span class="token punctuation">]</span> <span class="token operator">=</span> useState<span class="token operator">&lt;</span><span class="token keyword">undefined</span> <span class="token operator">|</span> number<span class="token operator">></span><span class="token punctuation">(</span><span class="token keyword">undefined</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">const</span> <span class="token punctuation">[</span>dragging<span class="token punctuation">,</span> setDragging<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token function">useState</span><span class="token punctuation">(</span><span class="token boolean">false</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre></div> <p>Then we can create our onMouseDown function. We want to trigger this when we click on the separator, so let's add our onMouseDown event to the divider.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"> <span class="token keyword">const</span> <span class="token function-variable function">onMouseDown</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token parameter">e<span class="token operator">:</span> React<span class="token punctuation">.</span>MouseEvent</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token function">setSeparatorXPosition</span><span class="token punctuation">(</span>e<span class="token punctuation">.</span>clientX<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">setDragging</span><span class="token punctuation">(</span><span class="token boolean">true</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">;</span> <span class="token keyword">return</span> <span class="token punctuation">(</span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span> <span class="token attr-name">className</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>splitView<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">LeftPanel</span></span> <span class="token attr-name">leftWidth</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>leftWidth<span class="token punctuation">}</span></span> <span class="token attr-name">setLeftWidth</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>setLeftWidth<span class="token punctuation">}</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token punctuation">{</span>left<span class="token punctuation">}</span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span><span class="token class-name">LeftPanel</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span> <span class="token attr-name">className</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>divider<span class="token punctuation">"</span></span> <span class="token attr-name">onMouseDown</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>onMouseDown<span class="token punctuation">}</span></span><span class="token punctuation">/></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span> <span class="token attr-name">className</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>rightPane<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token punctuation">{</span>right<span class="token punctuation">}</span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>div</span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>div</span><span class="token punctuation">></span></span> <span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre></div> <p>Next up is the onMouseMove event. If the flag isn't set we can just return. If the flag <em>is</em> set, then we're going to want to update leftWidth. The value to modify leftWidth by is the mouse's current position minus the stored mouse position. All we need to do is add that value to the leftWidth using the setter. We update the stored mouse position as well, so we can repeat this event for as long as the user is dragging the slider.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"> <span class="token keyword">const</span> <span class="token function-variable function">onMouseMove</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token parameter">e<span class="token operator">:</span> React<span class="token punctuation">.</span>MouseEvent</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>dragging <span class="token operator">&amp;&amp;</span> leftWidth <span class="token operator">&amp;&amp;</span> separatorXPosition<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">const</span> newLeftWidth <span class="token operator">=</span> leftWidth <span class="token operator">+</span> e<span class="token punctuation">.</span>clientX <span class="token operator">-</span> separatorXPosition<span class="token punctuation">;</span> <span class="token function">setSeparatorXPosition</span><span class="token punctuation">(</span>e<span class="token punctuation">.</span>clientX<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">setLeftWidth</span><span class="token punctuation">(</span>newLeftWidth<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span><span class="token punctuation">;</span></code></pre></div> <p>To trigger this function, we need event listeners. We can add those to the document using <code>document.addEventListener</code>. If we create a new useEffect for our SplitView component, then the listener will be added when the component mounts. As an added bonus we can return a function in the useEffect, which is run when the component unmounts, to remove the eventListeners and clean up after ourselves.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"> <span class="token function">useEffect</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> document<span class="token punctuation">.</span><span class="token function">addEventListener</span><span class="token punctuation">(</span><span class="token string">'mousemove'</span><span class="token punctuation">,</span> onMouseMove<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">return</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> document<span class="token punctuation">.</span><span class="token function">removeEventListener</span><span class="token punctuation">(</span><span class="token string">'mousemove'</span><span class="token punctuation">,</span> onMouseMove<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre></div> <p>Alright almost there. Finally we just need an onMouseUp function to reset the flag. We'll add eventListeners to the document for onMouseUp events too.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"> <span class="token keyword">const</span> <span class="token function-variable function">onMouseUp</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token function">setDragging</span><span class="token punctuation">(</span><span class="token boolean">false</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">;</span></code></pre></div> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"> <span class="token function">useEffect</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> document<span class="token punctuation">.</span><span class="token function">addEventListener</span><span class="token punctuation">(</span><span class="token string">'mousemove'</span><span class="token punctuation">,</span> onMouseMove<span class="token punctuation">)</span><span class="token punctuation">;</span> document<span class="token punctuation">.</span><span class="token function">addEventListener</span><span class="token punctuation">(</span><span class="token string">'mouseup'</span><span class="token punctuation">,</span> onMouseUp<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">return</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> document<span class="token punctuation">.</span><span class="token function">removeEventListener</span><span class="token punctuation">(</span><span class="token string">'mousemove'</span><span class="token punctuation">,</span> onMouseMove<span class="token punctuation">)</span><span class="token punctuation">;</span> document<span class="token punctuation">.</span><span class="token function">removeEventListener</span><span class="token punctuation">(</span><span class="token string">'mouseup'</span><span class="token punctuation">,</span> onMouseUp<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre></div> <p>And we're done! Congratulations... but at the moment this component just gets a passing grade from me. Let's see what we can do to make it nicer.</p> <h3>Improving the component</h3> <h4>Adding maximum and minimum widths</h4> <p>Because otherwise the component is <strong>really</strong> easy to break by losing the divider out of bounds and not being able to click on it.</p> <p>To stop the divider from doing this, let's add a minimum width for our panels with a const (if you wanted to get fancy you could pass it in as a prop). Then we just need a bit of extra logic in our onMouseMove function.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">const</span> <span class="token constant">MIN_WIDTH</span> <span class="token operator">=</span> <span class="token number">50</span><span class="token punctuation">;</span> <span class="token keyword">const</span> <span class="token function-variable function">onMouseMove</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token parameter">e<span class="token operator">:</span> React<span class="token punctuation">.</span>MouseEvent</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>dragging <span class="token operator">&amp;&amp;</span> leftWidth <span class="token operator">&amp;&amp;</span> separatorXPosition<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">const</span> newLeftWidth <span class="token operator">=</span> leftWidth <span class="token operator">+</span> e<span class="token punctuation">.</span>clientX <span class="token operator">-</span> separatorXPosition<span class="token punctuation">;</span> <span class="token function">setSeparatorXPosition</span><span class="token punctuation">(</span>e<span class="token punctuation">.</span>clientX<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>newLeftWidth <span class="token operator">&lt;</span> <span class="token constant">MIN_WIDTH</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token function">setLeftWidth</span><span class="token punctuation">(</span><span class="token constant">MIN_WIDTH</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">return</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token function">setLeftWidth</span><span class="token punctuation">(</span>newLeftWidth<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span><span class="token punctuation">;</span></code></pre></div> <p>So that takes care of the left edge, but what about the right? Well if we had the width of the whole component then we could calculate a maximum width for the left panel and use the logic above. Now how could we access the width of our component…</p> <p>That's right, using another ref! Let's attach it to the div wrapping all our components.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">const</span> splitPaneRef <span class="token operator">=</span> createRef<span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">HTMLDivElement</span></span><span class="token punctuation">></span></span><span class="token plain-text">(); ... return ( </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span> <span class="token attr-name">ref</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>splitPaneRef<span class="token punctuation">}</span></span> <span class="token attr-name">className</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>splitView<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">LeftPanel</span></span> <span class="token attr-name">leftWidth</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>leftWidth<span class="token punctuation">}</span></span> <span class="token attr-name">setLeftWidth</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>setLeftWidth<span class="token punctuation">}</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token punctuation">{</span>left<span class="token punctuation">}</span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span><span class="token class-name">LeftPanel</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span> <span class="token attr-name">className</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>divider<span class="token punctuation">"</span></span> <span class="token attr-name">onMouseDown</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>onMouseDown<span class="token punctuation">}</span></span><span class="token punctuation">/></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span> <span class="token attr-name">className</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>rightPane<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token punctuation">{</span>right<span class="token punctuation">}</span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>div</span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>div</span><span class="token punctuation">></span></span><span class="token plain-text"> );</span></code></pre></div> <p>Now our onMouseMove logic becomes:</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"> <span class="token keyword">const</span> <span class="token function-variable function">onMouseMove</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token parameter">clientX<span class="token operator">:</span> number</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>dragging <span class="token operator">&amp;&amp;</span> leftWidth <span class="token operator">&amp;&amp;</span> separatorXPosition<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">const</span> newLeftWidth <span class="token operator">=</span> leftWidth <span class="token operator">+</span> clientX <span class="token operator">-</span> separatorXPosition<span class="token punctuation">;</span> <span class="token function">setSeparatorXPosition</span><span class="token punctuation">(</span>clientX<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>newLeftWidth <span class="token operator">&lt;</span> <span class="token constant">MIN_WIDTH</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token function">setLeftWidth</span><span class="token punctuation">(</span><span class="token constant">MIN_WIDTH</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">return</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>splitPaneRef<span class="token punctuation">.</span>current<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">const</span> splitPaneWidth <span class="token operator">=</span> splitPaneRef<span class="token punctuation">.</span>current<span class="token punctuation">.</span>clientWidth<span class="token punctuation">;</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>newLeftWidth <span class="token operator">></span> splitPaneWidth <span class="token operator">-</span> <span class="token constant">MIN_WIDTH</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token function">setLeftWidth</span><span class="token punctuation">(</span>splitPaneWidth <span class="token operator">-</span> <span class="token constant">MIN_WIDTH</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">return</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> <span class="token function">setLeftWidth</span><span class="token punctuation">(</span>newLeftWidth<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span><span class="token punctuation">;</span></code></pre></div> <p>Perfect, no more disappearing dividers.</p> <h4>What is this? A divider for ants?</h4> <p>It's not ideal that our user has to click on an area 2 pixels wide if they want to resize the panels. Maybe we could make it wider?</p> <p align="center"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 450px; "> <a class="gatsby-resp-image-link" href="/static/4e863beb6f59d338f90c554d3ddfe557/fc2a6/big-divider.png" style="display: block" target="_blank" rel="noopener"> <span class="gatsby-resp-image-background-image" style="padding-bottom: 28.64864864864865%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAGCAYAAADDl76dAAAACXBIWXMAABYlAAAWJQFJUiTwAAAA4klEQVQY031RTQtFUBT0l5UtP0RWLGxYsJWFZMOGJRbKR1n4KoUU87r35XbV602dxTlzmjNzrwAO932zIkjTFL7vY9s2xv8D4YVlWVBVFfI8f4kSqKoKURQxDAPtz/NEkiToug5t26JpGtR1jb7vMc8z3RGKooBlWYiiiA6u62KChmFAURSM40j74zjgeR50XYdt2wjDEK7rwnEcxHH8FSQXCZFlGYtGRB9BWZYxTdMr8r7vP+PTyGVZUkESmTjgBTVNgyRJzCGZPxz/PHwq4deVhwyCAKZpYl3Xl0N+h/9IUh/LEb+CFGNVGQAAAABJRU5ErkJggg==&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Incorrect divider padding" title="Incorrect divider padding" src="/static/4e863beb6f59d338f90c554d3ddfe557/fc2a6/big-divider.png" srcset="/static/4e863beb6f59d338f90c554d3ddfe557/1d79a/big-divider.png 185w, /static/4e863beb6f59d338f90c554d3ddfe557/1efb2/big-divider.png 370w, /static/4e863beb6f59d338f90c554d3ddfe557/fc2a6/big-divider.png 450w" sizes="(max-width: 450px) 100vw, 450px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </a> </span> </p> <p>That looks terrible. How about padding it a little with another div?</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"> <span class="token keyword">return</span> <span class="token punctuation">(</span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span> <span class="token attr-name">className</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>container<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">LeftPanel</span></span> <span class="token attr-name">leftWidth</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>leftWidth<span class="token punctuation">}</span></span> <span class="token attr-name">setLeftWidth</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>setLeftWidth<span class="token punctuation">}</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token punctuation">{</span>left<span class="token punctuation">}</span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span><span class="token class-name">LeftPanel</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span> <span class="token attr-name">className</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>divider-hitbox<span class="token punctuation">"</span></span> <span class="token attr-name">onMouseDown</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>onMouseDown<span class="token punctuation">}</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span> <span class="token attr-name">className</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>divider<span class="token punctuation">"</span></span><span class="token punctuation">/></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>div</span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span> <span class="token attr-name">className</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>rightPane<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token punctuation">{</span>right<span class="token punctuation">}</span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>div</span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>div</span><span class="token punctuation">></span></span> <span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre></div> <div class="gatsby-highlight" data-language="css"><pre class="language-css"><code class="language-css"><span class="token selector">.divider-hitbox</span> <span class="token punctuation">{</span> <span class="token property">cursor</span><span class="token punctuation">:</span> col-resize<span class="token punctuation">;</span> <span class="token property">align-self</span><span class="token punctuation">:</span> stretch<span class="token punctuation">;</span> <span class="token property">display</span><span class="token punctuation">:</span> flex<span class="token punctuation">;</span> <span class="token property">align-items</span><span class="token punctuation">:</span> center<span class="token punctuation">;</span> <span class="token property">padding</span><span class="token punctuation">:</span> 0 1rem<span class="token punctuation">;</span> <span class="token punctuation">}</span></code></pre></div> <p align="center"> <img alt="Correctly styled divider" src="/2ffeb951faf060a1b9c085832680f246/improve-cursor.gif" style="margin-top:30px;margin-bottom:20px;width:60%;max-width:60%"> </p> <p>Muuuuch better. Changing the cursor also makes it much clearer to the user that this is resizeable!</p> <p>This is really important for touch screens where 2 pixel precision isn't going to make you many friends.</p> <h4>But none of my touch screen users can use the feature!</h4> <p>That's right! Because we only added <em>mouse</em> events. We're going to need some <em>touch</em> events too!</p> <p>TouchEvents are pretty similar to mouseEvents so we can reuse most of our code. But since touch screens can support many touches at once, touchEvents store an array of <em>touches</em>. Each touch has its own x and y positions, so we need to tweak our functions a little to account for this.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"> <span class="token keyword">const</span> <span class="token function-variable function">onTouchStart</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token parameter">e<span class="token operator">:</span> React<span class="token punctuation">.</span>TouchEvent</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token function">setSeparatorXPosition</span><span class="token punctuation">(</span>e<span class="token punctuation">.</span>touches<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">.</span>clientX<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">setDragging</span><span class="token punctuation">(</span><span class="token boolean">true</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">;</span> <span class="token keyword">const</span> <span class="token function-variable function">onMove</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token parameter">clientX<span class="token operator">:</span> number</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>dragging <span class="token operator">&amp;&amp;</span> leftWidth <span class="token operator">&amp;&amp;</span> separatorXPosition<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">const</span> newLeftWidth <span class="token operator">=</span> leftWidth <span class="token operator">+</span> clientX <span class="token operator">-</span> separatorXPosition<span class="token punctuation">;</span> <span class="token function">setSeparatorXPosition</span><span class="token punctuation">(</span>clientX<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>newLeftWidth <span class="token operator">&lt;</span> <span class="token constant">MIN_WIDTH</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token function">setLeftWidth</span><span class="token punctuation">(</span><span class="token constant">MIN_WIDTH</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">return</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>splitPaneRef<span class="token punctuation">.</span>current<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">const</span> splitPaneWidth <span class="token operator">=</span> splitPaneRef<span class="token punctuation">.</span>current<span class="token punctuation">.</span>clientWidth<span class="token punctuation">;</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>newLeftWidth <span class="token operator">></span> splitPaneWidth <span class="token operator">-</span> <span class="token constant">MIN_WIDTH</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token function">setLeftWidth</span><span class="token punctuation">(</span>splitPaneWidth <span class="token operator">-</span> <span class="token constant">MIN_WIDTH</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">return</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> <span class="token function">setLeftWidth</span><span class="token punctuation">(</span>newLeftWidth<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span><span class="token punctuation">;</span> <span class="token keyword">const</span> <span class="token function-variable function">onMouseMove</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token parameter">e<span class="token operator">:</span> MouseEvent</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>dragging<span class="token punctuation">)</span> e<span class="token punctuation">.</span><span class="token function">preventDefault</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">onMove</span><span class="token punctuation">(</span>e<span class="token punctuation">.</span>clientX<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">;</span> <span class="token keyword">const</span> <span class="token function-variable function">onTouchMove</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token parameter">e<span class="token operator">:</span> TouchEvent</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token function">onMove</span><span class="token punctuation">(</span>e<span class="token punctuation">.</span>touches<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">.</span>clientX<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">;</span> <span class="token keyword">const</span> <span class="token function-variable function">onMouseUp</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token function">setSeparatorXPosition</span><span class="token punctuation">(</span><span class="token keyword">undefined</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">setDragging</span><span class="token punctuation">(</span><span class="token boolean">false</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">;</span> <span class="token function">useEffect</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> document<span class="token punctuation">.</span><span class="token function">addEventListener</span><span class="token punctuation">(</span><span class="token string">'mousemove'</span><span class="token punctuation">,</span> onMouseMove<span class="token punctuation">)</span><span class="token punctuation">;</span> document<span class="token punctuation">.</span><span class="token function">addEventListener</span><span class="token punctuation">(</span><span class="token string">'touchmove'</span><span class="token punctuation">,</span> onTouchMove<span class="token punctuation">)</span><span class="token punctuation">;</span> document<span class="token punctuation">.</span><span class="token function">addEventListener</span><span class="token punctuation">(</span><span class="token string">'mouseup'</span><span class="token punctuation">,</span> onMouseUp<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">return</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> document<span class="token punctuation">.</span><span class="token function">removeEventListener</span><span class="token punctuation">(</span><span class="token string">'mousemove'</span><span class="token punctuation">,</span> onMouseMove<span class="token punctuation">)</span><span class="token punctuation">;</span> document<span class="token punctuation">.</span><span class="token function">removeEventListener</span><span class="token punctuation">(</span><span class="token string">'touchmove'</span><span class="token punctuation">,</span> onTouchMove<span class="token punctuation">)</span><span class="token punctuation">;</span> document<span class="token punctuation">.</span><span class="token function">removeEventListener</span><span class="token punctuation">(</span><span class="token string">'mouseup'</span><span class="token punctuation">,</span> onMouseUp<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">return</span> <span class="token punctuation">(</span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">Container</span></span> <span class="token attr-name">ref</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>splitPaneRef<span class="token punctuation">}</span></span> <span class="token attr-name">className</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>className<span class="token punctuation">}</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">LeftPanel</span></span> <span class="token attr-name">leftWidth</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>leftWidth<span class="token punctuation">}</span></span> <span class="token attr-name">setLeftWidth</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>setLeftWidth<span class="token punctuation">}</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token punctuation">{</span>left<span class="token punctuation">}</span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span><span class="token class-name">LeftPanel</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span> <span class="token attr-name">className</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>divider-hitbox<span class="token punctuation">"</span></span> <span class="token attr-name">onMouseDown</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>onMouseDown<span class="token punctuation">}</span></span> <span class="token attr-name">onTouchStart</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>onTouchStart<span class="token punctuation">}</span></span> <span class="token attr-name">onTouchEnd</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>onMouseUp<span class="token punctuation">}</span></span> <span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span> <span class="token attr-name">className</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>divider<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>div</span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span> <span class="token attr-name">className</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>rightPane<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token punctuation">{</span>right<span class="token punctuation">}</span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>div</span><span class="token punctuation">></span></span><span class="token plain-text"> ); };</span></code></pre></div> <p>We can move out the core logic of the onMouseMove function into one shared onMove function that both the onTouchMove and onMouseMove functions can use. Also note that we use the onTouchStart and onTouchEnd events, both attached to the Separator, and reuse the onMouseUp function for onTouchEnd.</p> <p>One last note is the addition of e.preventDefault if dragging is true in onMouseMove. This stops text being highlighted (or other onDrag mouse events from being triggered) while moving the divider around.</p> <h4>That's great! Can I style it too?</h4> <p>You can! The final touch to our component will be adding a className prop. Drop the component anywhere in your project and style it to your heart's content using your method of choice, whether it be stylesheets, styled-components or something else.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">interface</span> <span class="token class-name">SplitViewProps</span> <span class="token punctuation">{</span> left<span class="token operator">:</span> React<span class="token punctuation">.</span>ReactElement<span class="token punctuation">;</span> right<span class="token operator">:</span> React<span class="token punctuation">.</span>ReactElement<span class="token punctuation">;</span> className<span class="token operator">?</span><span class="token operator">:</span> string<span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token keyword">export</span> <span class="token keyword">const</span> SplitView<span class="token operator">:</span> React<span class="token punctuation">.</span>FunctionComponent<span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">SplitViewProps</span></span><span class="token punctuation">></span></span><span class="token plain-text"> = ({ left, right, className }) => </span><span class="token punctuation">{</span> <span class="token operator">...</span> <span class="token keyword">return</span> <span class="token punctuation">(</span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span> <span class="token attr-name">ref</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>splitPaneRef<span class="token punctuation">}</span></span> <span class="token attr-name">className</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span><span class="token template-string"><span class="token template-punctuation string">`</span><span class="token string">splitView </span><span class="token interpolation"><span class="token interpolation-punctuation punctuation">${</span>className <span class="token operator">??</span> <span class="token string">""</span><span class="token interpolation-punctuation punctuation">}</span></span><span class="token template-punctuation string">`</span></span><span class="token punctuation">}</span></span><span class="token punctuation">></span></span><span class="token plain-text"> ></span></code></pre></div> <p>Happy resizing!</p> <p><span>(Cover photo by <a href="https://unsplash.com/@fabrizioverrecchia">Fabrizio Verrecchia</a> on <a href="https://unsplash.com/s/photos/window">Unsplash)</a></span></p><![CDATA[Manage your technical debt roadmap right from your code 🚀]]>/2020/10/typescript-technical-roadmap-debt-sonarqube-tyrion//2020/10/typescript-technical-roadmap-debt-sonarqube-tyrion/Wed, 21 Oct 2020 00:00:00 GMT<style> a { font-weight: normal; } code { font-family: "SFMono-Regular", Consolas, "Liberation Mono", Menlo, Courier, monospace; line-height: normal; background: rgba(135,131,120,0.15); color: #eb5757; border-radius: 3px; padding: 0.2em 0.4em; font-size: 0.7rem; } .boxes { display: flex; align-items: flex-start; } .boxes .box {} .boxes.row { flex-flow: row nowrap; } .boxes.row > * { width: 10%; flex-grow: 1; } .boxes.row > *:not(:first-child) { margin-left: 16px; } .boxes.column { flex-flow: column nowrap; } .box { padding: 16px 16px 16px 44px; width: 100%; border-radius: 3px; border-width: 1px; border-style: solid; border-color: transparent; background: rgba(235, 236, 237, 0.3) none repeat scroll 0% 0%; position: relative; font-size: 0.9em; margin: 8px 0 1.45rem; } .box.blue { background: rgba(221, 235, 241, 0.3) none repeat scroll 0% 0%; } .box a { font-weight: normal; } .box ol:last-child { margin-bottom: 0; } .box li { margin-bottom: 0; } .box aside { position: absolute; top: 16px; left: 12px; font-size: 1.2em; } .wide-image-container { display: flex; justify-content: center; height: min(60vw, 600px); } .wide-image-container img { position: absolute; width: 1067px; } li { margin-bottom: 0; } p { margin-bottom: 1rem; } figcaption { text-rendering: optimizeLegibility; -webkit-font-smoothing: antialiased; word-break: break-word; box-sizing: inherit; font-family: "Lucida Grande", "Lucida Sans Unicode", "Lucida Sans", Geneva, Arial, sans-serif; font-weight: 300; font-size: 16px; line-height: 20px; color: rgba(117, 117, 117, 1); margin-right: auto; max-width: 728px; margin-top: 10px; margin-left: auto; text-align: center; display: block; } blockquote { border-left: 0.5em solid #DDD; padding: 10px 10px 10px 20px; box-shadow: 0 0 6px rgba(0,0,0,0.5); } blockquote footer { padding-top: 10px; } li blockquote { margin-left: 0; } blockquote p { margin: 0; } blockquote p+p { margin-top: 1.5em; } blockquote footer { margin-bottom: 0; } blockquote blockquote, .callout blockquote, blockquote ol, blockquote ul { margin-left: 1.5em; margin-right: .75em; } blockquote>blockquote { margin-top: .75em } .note-block blockquote { font-style: normal; } blockquote mark { font-weight: bold; } blockquote strong mark, blockquote mark strong { font-style: italic; } /* Quotes for two levels of nested quotations */ q { quotes: '“' '”' '‘' '’'; } /* extra content definitions for pre-2011 WebKit */ q:before { content: '“'; content: open-quote; } q:after { content: '”'; content: close-quote; } q q:before { content: '‘'; content: open-quote; } q q:after { content: '’'; content: close-quote; } /* q in blockquote */ blockquote q:before { content: '‘'; content: open-quote; } blockquote q:after { content: '’'; content: close-quote; } blockquote q q:before { content: '“'; content: open-quote; } blockquote q q:after { content: '”'; content: close-quote; } /* hanging opening quote */ blockquote:before { display: block; height: 0; content: "“"; margin-left: -.95em; font: italic 400%/1 Cochin,Georgia,"Times New Roman", serif; color: #999; } </style> <p><strong>Exec summary:</strong></p> <ul> <li>Technical debt leads to bugs creation: the number of bugs and the number of design flaws (technical debt) are 0.92 correlated in a <a href="https://www.usenix.org/system/files/conference/cset16/cset16-paper-nord.pdf">study from the Software Engineering Institute</a></li> <li>Static analysis tools like SonarQube are massively used (100k+ users) to chase quality defects, but they can't replace human intelligence</li> <li>Bug tracking system are time-consuming and don't help developers while they code</li> <li>Our tool <a href="https://github.com/theodo/tyrion">Tyrion</a> fixes these problems and allows developers to see quality problems right in their IDE</li> </ul> <h2>Quality is worth its investment</h2> <p>Martin Fowler in his famous <a href="https://martinfowler.com/articles/is-quality-worth-cost.html">article on quality</a> explains why investing in software quality is cheaper than not investing in it.</p> <figure> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <span class="gatsby-resp-image-background-image" style="padding-bottom: 47.56756756756757%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAKCAYAAAC0VX7mAAAACXBIWXMAAAsTAAALEwEAmpwYAAACCklEQVQoz2NgwAEki3ZB6Z0MksW7EfyCbdySBdtFJAt3gLC0VNFOAcmSPVxAzCdZvIsDpEYeiEVADAVuBQY3aS+wRpkioCHFexgkS/fDLZEt2MnIV3+cjafxFKdA7TEunoaTHAzH/jNKlu5lQnYMDxTDgXThLhTXKpRsZ5Ep3sUtWbaHxyh2EU+QbQNXqVwMywQWB5ZSxVh2hMr/CGaGayGDb/ciuJBV2XIGuZKdHILFB3lYik9xMqTfZAaJ54rEC2TIp6mkKWaopipmykTwZHODxC8urmG4sagZqEAtRz5FJVUmQzZVKksiSSpYrZyboeshm2jhPl7W4lPsJeW1LOeq7Fl2VvqyzqtMYJtdmcS9oDKBZ1ZRPO+SugSepc0xQnPL40SBYmJLapK4GHIVspUzRGPE8gLKxaKCGhW8MqaIZVc0Cs4ujeXdWObHPb80imtaaQLnzNI4ztmlMZxTCyOZG9I8GNuKXBj7ssOYZhTGCMwsieGfWRLNP6UggoPh+IX9bOlNk0XTO+dIV8+cy/1gfjYbAyXAPrpEmkHXjcs3zJMXyAWFEwsQs8MwEyMjOwcbKzs7Kws7OxsLmA+TY2RkYOfmYGPn4mDj4OFk52BhZgKHJ6sZgypfpE20rKuCq4g+vy4vWDEDQiMBDPIRyCAxIAanQ4Z89QLRXKUcmTzVXOki9UIBSnwMAOSKiBVKOhbeAAAAAElFTkSuQmCC&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Why quality in software is cheaper than no quality: Software with high internal quality gets a short initial slow down, but delivers more rapidly later" title="Why quality in software is cheaper than no quality: Software with high internal quality gets a short initial slow down, but delivers more rapidly later" src="/static/d97a1fdf6f7b5d0029d8888f6a137f78/50383/debt-quality-speed.png" srcset="/static/d97a1fdf6f7b5d0029d8888f6a137f78/1d79a/debt-quality-speed.png 185w, /static/d97a1fdf6f7b5d0029d8888f6a137f78/1efb2/debt-quality-speed.png 370w, /static/d97a1fdf6f7b5d0029d8888f6a137f78/50383/debt-quality-speed.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </span> <figcaption>Why quality in software is cheaper than no quality</figcaption> </figure> <p>This knowledge has also long been known for lean practitioners: "<em>If you focus on quality, you will deliver a high quality product on time. If you focus on delivery, you will deliver a low quality product late.</em>"</p> <p>Why is that? If you focus on delivering things <strong>right the first time</strong> you will have less rework to do and then save time in the long run. Teams that don't do that end up spending most of their time fixing bugs.</p> <figure> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <span class="gatsby-resp-image-background-image" style="padding-bottom: 20%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAECAYAAACOXx+WAAAACXBIWXMAAAsSAAALEgHS3X78AAABJUlEQVQY0yWPS0sCcRTFp0WLFkVQ9ESa8FFCSZRapqm9izERLYkKssHxbWSFCW2yLAwiSKGdC4MI3AVFiz7cr7/TXdz3PedcqdH6ZcQTZ2Ijj7ycRi3VUeIVxvwpLOs57OEixccmtkAB01pW7+8kH/S9dj65mcfgTfD6/kPbpLa7qX3Q74xhEQeDCyrB5D025YxxfxrZl9SPItmqqFOYVjOMuuO49kr4Dq4Z9Wiil6V75oinxuc/4EvzS1+0bp0y5FIJZ6rMhS4Fs4ZhKYFRzA7Pn5lWCjiF4vY3XgGmaBVBlhPK03Rao5SFMKn+9o0kB5H9SYwrabbVMrOhCzrMEXodYYYXY5xc1ehzHNM1ta/H+UiRgHaHZN5lwB2lx65wW2/pL/8BylymZfxLh2MAAAAASUVORK5CYII=&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Low quality leads to too many bugs and missed deadlines" title="Low quality leads to too many bugs and missed deadlines" src="/static/4ada7940719c813310c8d6ba9aece1b1/50383/bugs-flow.png" srcset="/static/4ada7940719c813310c8d6ba9aece1b1/1d79a/bugs-flow.png 185w, /static/4ada7940719c813310c8d6ba9aece1b1/1efb2/bugs-flow.png 370w, /static/4ada7940719c813310c8d6ba9aece1b1/50383/bugs-flow.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </span> <figcaption>Low quality leads to too many bugs and missed deadlines</figcaption> </figure> <h2>Static analysis tools are mandatory but not enough to monitor quality</h2> <p>Once you know you want to deliver quality on the long run, there are a lot of things to do: identifying best practices, tech trainings, problem solving, quality tooling, setting up all kind of monitoring, choosing quality KPIs ...</p> <p>If you are serious about keeping the quality of the codebase high, you need to proactively chase, monitor and resolve code quality defects. Even if you have set up all the best linters rules and have a highly trained team, in the long run you will have defects. Indeed, there are a lot of defects that linters can't detect: non homogeneous code and architecture, wrong patterns usage, naming not coherent with business...</p> <p>A widespread strategy is to monitor quality by static analysis tools like <a href="https://www.sonarqube.org/">SonarQube</a> or <a href="https://codeclimate.com/">CodeClimate</a>. These tools have some advantages:</p> <ul> <li>✔️You get a quality score without extra work from a developer (once the tool has been set up which can be quick)</li> <li>✔️Results are objective</li> <li>✔️They give you a list of piece of code you need to change and which good practices have not been followed</li> </ul> <p>You can have graphs like this one which give you the trend of your technical debt and show you if the situation is getting better or worse.</p> <figure> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <span class="gatsby-resp-image-background-image" style="padding-bottom: 55.13513513513514%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAALCAYAAAB/Ca1DAAAACXBIWXMAAAsSAAALEgHS3X78AAABZElEQVQoz4WSCW/CMAyF+///3hgM2KQdbNM4So8kzdm3l6RlFBCLZLly48/PdooQAuLp+z55bR2quobzIfnDfg/jHGTXYbvdom1bjGdI4V3PuIA1CoW19gQcoZPbZ8XOj7UaTn3Cyw08fej4rfcoLi+Oyf1FkegN1YvO0Cxa2aLaPUId55DlAqqcwYn3a4WXFo9n+43UUNqk9nxgPBBcLgmdozk8Yff9ACM2WeE9WFQVYX6YddZOnmfL7RquWSGINWqq1e3HfYW5PZPhk38ZaJo1dL2EpS9/ZlANW75axgBvlU4bv5rt6AmMIJOAK85xjvr4isK5nGSdhzZ8HpxTJboTLNwcx59CMyhUxwXq8i0CPRTb0sYmiwuYPp1bFjfFGRJkCYxz1AkYFRIgqEwR7AiwHL7x/xi3rPkOZf1Ce05eVJyn/IpAD2mo0hHc8WlQpWVCTswxzWLmVMinWCrs7MSsd/gFmA5cS5H+GqkAAAAASUVORK5CYII=&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="A SonarQube debt graph from a current project" title="A SonarQube debt graph from a current project" src="/static/c4d91951e26873df7bb829006439a467/50383/sonarGraph.png" srcset="/static/c4d91951e26873df7bb829006439a467/1d79a/sonarGraph.png 185w, /static/c4d91951e26873df7bb829006439a467/1efb2/sonarGraph.png 370w, /static/c4d91951e26873df7bb829006439a467/50383/sonarGraph.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </span> <figcaption>A SonarQube debt graph from a current project</figcaption> </figure> <p>When talking with tech leads and CTOs, many of them told me that SonarQube has been efficient to help their team keep the quality of their code high.</p> <blockquote> <cite> Static Code Analysis Tools (SCAT) provide objective metrics and insights of the code quality and technical debt. However, these tools require a real integration effort. Such tools without a team adoption and training are of little value. On the opposite, used by trained developers, with a pragmatic understanding of the metrics and warnings, these tools offer a lot of clues on the next refactoring needed to reduce project entropy, on the security breaches still to address and on the potential bugs to review. SCAT warnings and suggestions must be used as starting points of developers' discussions around quality standards. SCAT must not be considered as referees stating good/bad quality points of a project or a team. </cite> <footer>— Guillaume Michel, Theodo, VP Engineer</footer> </blockquote> <p>It has also been proven to be effective in some studies like <a href="https://arxiv.org/pdf/1910.12816.pdf">Oswal, Nikhil. "Technical Debt: Identify, Measure and Monitor." (2019)</a></p> <p>However, those tools have also limitations:</p> <ul> <li>❌ they create quite a lot of false positive (labeling a piece of code as bad when it is good) and false negative (missing some bad piece of code) requiring times to sort everything.</li> <li>❌ they can't inspect all aspects of a project. For example some defects like a wrong or non homogeneous directories' organisation are never detected</li> </ul> <p>Academic research while confirming the positive impact of static analysis tools insist that they are not mature enough to only rely on them. For example, to assess if a defect will create a bug, Valentina Lenarduzz warns us in her 2019 study "<a href="https://arxiv.org/pdf/1907.00376.pdf">Are SonarQube Rules Inducing Bugs?</a>": "<em>SonarQube violation severity is not related to the fault-proneness and there-fore, developers should carefully consider the severity as decision factor for refactoring a violation.</em>"</p> <p>This is exactly what Bill Clark explained very well in his article "<a href="https://technology.riotgames.com/news/taxonomy-tech-debt">A TAXONOMY OF TECH DEBT</a>", human can provide much more valuable information about a piece of code with technical debt like the <strong>contagious</strong> aspect: "Is it likely this bad code practice could spread somewhere else by copy/paste?"</p> <h2>Performant quality monitoring requires human analysis</h2> <p>As we just see, human analysis provides complementary information to static analysis tools:</p> <ul> <li>✔️human can analyse every aspect of the project: architecture, naming, easiness of understanding, maturity of library and frameworks</li> <li>✔️they can provide much more information about the problems they detect since they understand the context around the codebase</li> <li>✔️they can provide a prioritization of the problems</li> </ul> <p>The more the developers in your teams are trained about writing quality the more efficient they will be to detect defects. This is another reason why you should invest a lot in training if you want to achieve high quality in your code.</p> <p>You also need to agree on the best way to write things: what is OK and what is not OK to write in the code. For example, these pieces of code are doing the exact same thing in Symfony.</p> <div class="gatsby-highlight" data-language="php"><pre class="language-php"><code class="language-php"><span class="token comment">/** * @ORM\Column(unique=true) */</span> <span class="token keyword">private</span> <span class="token variable">$email</span><span class="token punctuation">;</span> <span class="token comment">/** * @ORM\Column(type="string", length=255, unique=true) */</span> <span class="token keyword">private</span> <span class="token variable">$email</span><span class="token punctuation">;</span></code></pre></div> <p>Both ways have their advantages: the former is shorter to write/read and the latter is more explicit. I prefer the explicit version because I think in this case the tradeoff between the code being super clear and having more words to write and read is worth it. Some other people could say that it is not worth it to add the type and length because it's the default.</p> <p>Who is right is less important than having your team agreeing in the standard way of doing it. Because once you agreed, people are autonomous to write and check quality on their own regarding this standard making little by little the whole code base coherent.</p> <p>The main drawback of human analysis is that they require a lot of effort to get and update the information about the overall quality.</p> <figure> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 543px; "> <span class="gatsby-resp-image-background-image" style="padding-bottom: 43.78378378378379%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAJCAYAAAAywQxIAAAACXBIWXMAAAsSAAALEgHS3X78AAACS0lEQVQoz0WS609SYRjA+Sf72Ifqg12Gl602G+IFc+VlooAoxmSFhaJia6vMK07ayk2EwzkgBzhcB5pGnBJow/r14tb68Nvzfnj2vL/nYmi1WlxdXdFqI97/qF9eIskSiWScpJpAUmKciKjlNMrlEmpcQUulSMbjKDGZ6vkFp+UyhkRCJEsxUqqKKhIURaFYLFHVv+HYGMH6sR/7joXprUEc2xasQQuB5VFUq53UrJPs3BwZp5OC203S78cQiUQ4OgqTaP+kxIknTqh91ymWCwz7HmIJ9DC1a8a604d1u4/xoAmvx8Rn8xDyxDgl5wwFh4O83U7c68WQTqfJ5fLIMQlZVkR7qih+Qjqbwv1pDPvBAPZ9YSeYEUwe9PN25Sn6/AKV5/OUXS4qgnZUfT5hGJU4Dh+TLxTEbMpUKhVqNR29XmP0jYlnG73YDoawimJTgvHQAEuLwu7JGJlpG1/n5jmbdV2Tet0uGImixBOkMxkKhTyappHRclRKOQKODlanbxLyGYmI1sNr3cQC3QQ8RoatD3Av9fFq3YI3MMSiIPBuFkNUGKbTGbEY6RpFlpHFB0k5yqn/EeeLd9H9ndRXu/gpaK10sicY3OxlMmhmYv8/nn0HhphYua7rNBoN6vU6zWaTRvMX55Ui0oIR2XWL8pIRfb2HqrBrrHezJYxNy93Yts3YgwPY9sScBd6gE0M4HEYSRb8cHhIKhchqWX7/gR/VM3Yn77A5cgPV00FtrYsLYXYpLN+/vMd9x20mPjwWF9AvLsB8zYu9Gf4C9Vw+XRFpzmQAAAAASUVORK5CYII=&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Comparison between static analysis and human analysis for software quality. Static analysis requires less effort than human but provides less information" title="Comparison between static analysis and human analysis for software quality. Static analysis requires less effort than human but provides less information" src="/static/cd94e46801dd3a80ebc9a083127ec72e/29579/static-analysis-vs-human.png" srcset="/static/cd94e46801dd3a80ebc9a083127ec72e/1d79a/static-analysis-vs-human.png 185w, /static/cd94e46801dd3a80ebc9a083127ec72e/1efb2/static-analysis-vs-human.png 370w, /static/cd94e46801dd3a80ebc9a083127ec72e/29579/static-analysis-vs-human.png 543w" sizes="(max-width: 543px) 100vw, 543px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </span> <figcaption>Comparison between static analysis and human analysis for software quality</figcaption> </figure> <h2>Existing tools are time-consuming and don't help devs when they are coding</h2> <p>At Theodo we experienced several systems to track self admitted technical debt (SATD, as <a href="https://scholar.google.fr/scholar?q=self+admitted+technical+debt&#x26;hl=en&#x26;as_sdt=0&#x26;as_vis=1&#x26;oi=scholart">software quality researchers calls it</a>).</p> <ol> <li>On small or short projects, we use a special column in Trello/Jira/Monday/Google doc to track tasks we need to do to fix quality problems. This works OK as long as the team fixes everything quickly, so the column doesn't get too long and messy.</li> <li>When the project grows we quickly need a real tech roadmap on a separate board.</li> </ol> <figure> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <span class="gatsby-resp-image-background-image" style="padding-bottom: 44.32432432432433%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAJCAIAAAC9o5sfAAAACXBIWXMAAAsSAAALEgHS3X78AAABq0lEQVQoz22OWS8DcRTFby2JxIOIxBeoeJTgWxFbotWxL0+mieXBkxAPHizVFkXQzn9KVaRFR6qWms5ma1Ft6cyoMmYIIZr8cnNy7jm5F2pwVI0TtWZUaybLTI7CFnt+sx0aVWzQaNVmk011io0OaLBqaL7tywd8jbL46DkfPbVzNrpBLfgZi5+ZcIeG1wMWTbPj5MkkGdykGBTk7fvMmJOa2buY9THT3nOIsJtiJixkeDF5GhYERU4qcip+y56zkTdR1enH2H3PEoUtHvWuHNfNUVSEVpTMu/KalVJwQNOilM6+Z++fJPJY4G7uhGj8iH84pFkx/SzLLzfRK33/FtRvFDQ7oW69czm0SvHWfW7+gIcQxysZWXmVH5+lStxT2usuH3CX9ZGrgXBW0spXd7FKswfaiMJOMr+DhDaXqj9xwaQn7ApyKHRpP+RLut1gcOmMBBgJR+DsTUq9SPJ1Iq7HPWAg8toRYH9R0xqtKk4dhvIwpPtcWPxcIpGIxVNcNKnHvWomR1nXjn74vagY3K4Z8VYP71YNeYu6SDD9a2qXMZQb9XnDN6bcmQ+u1EHkEJHgEgAAAABJRU5ErkJggg==&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Example of debt management using Trello" title="Example of debt management using Trello" src="/static/407f5f0b7c22eab75ba0bc229a62e357/50383/trello_debt_management.png" srcset="/static/407f5f0b7c22eab75ba0bc229a62e357/1d79a/trello_debt_management.png 185w, /static/407f5f0b7c22eab75ba0bc229a62e357/1efb2/trello_debt_management.png 370w, /static/407f5f0b7c22eab75ba0bc229a62e357/50383/trello_debt_management.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </span> <figcaption>Example of debt management using Trello</figcaption> </figure> <p>We can also use a specialized bug tracking tool like <a href="https://www.bugzilla.org/">Bugzilla</a> or <a href="https://bugherd.com/">Bugherd</a>. It works well, but it has two drawbacks:</p> <ul> <li>❌ it is time-consuming to update it when an item has been fixed or is no longer valid</li> <li>❌ developers don't see the defect in their IDE while coding and thus could copy/paste it creating the contagious aspect of the debt explained by Bill Clark.</li> </ul> <h2>Introducing Tyrion: a tool to help teams track software quality through human analysis</h2> <p>I initiated <a href="https://github.com/theodo/tyrion">Tyrion</a> to remove those main drawbacks and be able to:</p> <ul> <li>✔️Document the debt while coding directly in the IDE</li> <li>✔️Painlessly update the debt tracking system when something has been fixed</li> <li>✔️Quickly get an overview and the trend of the debt</li> </ul> <h3>Writing comments in the code to map the debt</h3> <p>Instead of registering all technical defects in a tool, the main idea of Tyrion is to add them directly in the code base as comments. Tyrion will parse these comments to generate the debt graphs or CSV files so you can analysis the debt.</p> <p>The syntax of the comment is straightforward: the only thing you need to add to a classic comment is a type to allow Tyrion to group them:</p> <div class="gatsby-highlight" data-language="typescript"><pre class="language-typescript"><code class="language-typescript"><span class="token comment">// TODO DEBT_TYPE "Author: comment"</span></code></pre></div> <p>I like the following example from a current project. Albéric and Emyly are working on improving the accessibility of the webapp. Nothing urgent from the business and nothing requires immediate change from a technical point of view. But still, they know they can do this piece of code better, so they mark it as debt to improve it a bit later.</p> <div class="gatsby-highlight" data-language="typescript"><pre class="language-typescript"><code class="language-typescript"><span class="token comment">// TODO accessibility "Alberic: Add keyboard behaviours and focusability"</span> <span class="token keyword">export</span> <span class="token keyword">const</span> Select<span class="token operator">:</span> React<span class="token punctuation">.</span><span class="token constant">FC</span><span class="token operator">&lt;</span>Props<span class="token operator">></span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token punctuation">{</span></code></pre></div> <p>Even if you don't use Tyrion, adding debt comments in the code is valuable by it-self. It trains developers to evaluate the quality of the code while they are coding. It also partially protects you again the contagious effect of the debt because if you have correctly marked with comments the debt in the code, developers will think twice before copy/pasting a piece of code that is marked as a debt.</p> <p>If you have a large code base, it can take a while to document the whole technical debt. You can do it gradually during few weeks. While working on some part of the app each member of your team will read different files and find something to mark as debt. By doing that you will create little by little the debt map of your project. Of course, it's better to fix a problem when you see it but for various reason you don't always have the time to do it.</p> <p>If your project is of reasonable size, you can also organize a big debt mapping session one afternoon.</p> <h3>Using the CLI to generate the first graph</h3> <p>Once you have commented at least a part of the debt you have in your project, you can start using the Tyrion CLI.</p> <p>First you need to download it with: <code>npm i -g tyrionl</code> or <code>yarn global add tyrionl</code></p> <p>Then just run <code>tyrion</code> in the root directory of your project. It will scan your files to look for <code>TODO</code>, <code>FIXME</code> or <code>@debt</code> strings in the comments and then assume it is a debt comment.</p> <p>This first mode is useful to get information of the current debt status of your project. It can help you respond to question such as "How much debt do we have right now?" or "What <strong>kind</strong> of debt do we have most?"</p> <p>If I run Tyrion in the frontend part of our webapp, I got the following graph where we can find the <code>acessibility</code> type of debt from the above debt comment among other types we use on the project.</p> <figure> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <span class="gatsby-resp-image-background-image" style="padding-bottom: 41.62162162162162%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAICAYAAAD5nd/tAAAACXBIWXMAAAsSAAALEgHS3X78AAABEUlEQVQoz52SzUvDQBDF9+8XPKgXoVBLvQt+QU1TQw+i9aAXxc9iLCltTZFik+xuk+xzZ2pC1R6qA0PebnZ/+x6MkFIiz3Nuqmn0gYvLNh4eb3hd/Fu1hVKKRZZlDHgLB9iurKHh7PGa9v8EjOOYLxXAcDxCbXcLLe/4f8A0Tb9FHltgtb4B1zsqIxtjVm6RyIRdEJgdhkML3IR7evjL4TxJWu4tcy+01paMsijyjgW2vhwWl8zCIdJzgFkWOYO0LskZfSlypboOx93HYhH4xX/Cc/cOs5lm6M/H6AFBejDsw2ke2MP3mEzecWL11XUH5N73uwiCHpRWODv34LUbSJIYQf/VjtYt6yia2ukYgUbwEwW+ZAY3xSmmAAAAAElFTkSuQmCC&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="tyrion simple" title="tyrion simple" src="/static/f64520fc374968fb6a43f748fd426543/50383/tyrion-simple.png" srcset="/static/f64520fc374968fb6a43f748fd426543/1d79a/tyrion-simple.png 185w, /static/f64520fc374968fb6a43f748fd426543/1efb2/tyrion-simple.png 370w, /static/f64520fc374968fb6a43f748fd426543/50383/tyrion-simple.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </span> <figcaption>An example of the debt graph showing the current distribution of types of debt</figcaption> </figure> <h3>You can customize the weight of each type on the global debt score</h3> <p>All kind of debt doesn't have the same impact. A security issue can be more critic than an over complex function. So you may want to weight differently each kind of debt. This can be done by creating a <code>.tyrion-config.json</code> file in the root of your project. You can then assign different value to each type of debt.</p> <p>Here is the default configuration:</p> <div class="gatsby-highlight" data-language="json"><pre class="language-json"><code class="language-json"><span class="token punctuation">{</span> <span class="token property">"pricer"</span><span class="token operator">:</span> <span class="token punctuation">{</span> <span class="token property">"bug"</span><span class="token operator">:</span> <span class="token number">100</span><span class="token punctuation">,</span> <span class="token property">"architecture"</span><span class="token operator">:</span> <span class="token number">100</span><span class="token punctuation">,</span> <span class="token property">"bugRisk"</span><span class="token operator">:</span> <span class="token number">5</span><span class="token punctuation">,</span> <span class="token property">"security"</span><span class="token operator">:</span> <span class="token number">100</span><span class="token punctuation">,</span> <span class="token property">"securityRisk"</span><span class="token operator">:</span> <span class="token number">10</span><span class="token punctuation">,</span> <span class="token property">"quality"</span><span class="token operator">:</span> <span class="token number">5</span><span class="token punctuation">,</span> <span class="token property">"test"</span><span class="token operator">:</span> <span class="token number">5</span><span class="token punctuation">,</span> <span class="token property">"doc"</span><span class="token operator">:</span> <span class="token number">3</span><span class="token punctuation">,</span> <span class="token property">"ci"</span><span class="token operator">:</span> <span class="token number">30</span><span class="token punctuation">,</span> <span class="token property">"deploy"</span><span class="token operator">:</span> <span class="token number">10</span><span class="token punctuation">,</span> <span class="token property">"devEnv"</span><span class="token operator">:</span> <span class="token number">10</span><span class="token punctuation">,</span> <span class="token property">"outdated"</span><span class="token operator">:</span> <span class="token number">5</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token property">"standard"</span><span class="token operator">:</span> <span class="token number">100</span><span class="token punctuation">,</span> <span class="token property">"ignorePath"</span><span class="token operator">:</span> <span class="token punctuation">[</span> <span class="token string">"node_modules"</span><span class="token punctuation">,</span> <span class="token string">"README.md"</span> <span class="token punctuation">]</span> <span class="token punctuation">}</span></code></pre></div> <div class="blue box"> <aside>💡</aside> You can find more information on how to use Tyrion on the <a href="https://github.com/theodo/tyrion">README</a> </div> <h3>The trend of the debt may be the most important information</h3> <p>After a while of commenting debt in your code, you will be able to use the second mode that will analyse your Git history and will create a graph of the evolution of your debt. The command to get it is <code>tyrion -e 200</code>. You can replace <code>200</code> with the number of days you want to scan backward.</p> <figure> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <span class="gatsby-resp-image-background-image" style="padding-bottom: 48.64864864864865%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAKCAYAAAC0VX7mAAAACXBIWXMAAAsSAAALEgHS3X78AAABYklEQVQoz21Si47jIAzM/3/oNV01EN5gILO206SV9pBGYwZjjO1lXVes6wOPxwMxRsg6juMG73Ctb/1/kLXMOdF7x8UXiAhjDLRGcLEh5qa6aCd/7EuXoEvv477cmT8OHaV2/NgMzwGfJmNzRc9yaQip8WP9b8AxJkiEO8PB2Q7N6mkrGnV2nKrHzNkmgvUVge1U3onw2ScgNVAtEO61oreK0Ro2E+F9xqSq+mgnZqdTY9vuSbWD74ouZVx68GhuhzD5HSN6VN77l8URzzPyTtGEb38Hxz7buimST2dT9KuSNnPimtlAMIzM9uQn5Ux93n7jiwWFS1KI+8DflolYiIWQOH1fYLjorz1zbbgMUgJuEn2jtXsC6GqgMn1qaPaIf88XYkrYrUEpWefRWqu2sA9BNeccawXb9kJgzRijEF39vMciHTxx6BfOIZXvHBwocXOMFt7xYyGcg39lIzzfdy77F3KGDWxm3gOkAAAAAElFTkSuQmCC&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="An example of a debt report showing the evolution of the debt" title="An example of a debt report showing the evolution of the debt" src="/static/2b64993e0325ad4805daa76a8dcc91a3/50383/tyrion-report-evolution.png" srcset="/static/2b64993e0325ad4805daa76a8dcc91a3/1d79a/tyrion-report-evolution.png 185w, /static/2b64993e0325ad4805daa76a8dcc91a3/1efb2/tyrion-report-evolution.png 370w, /static/2b64993e0325ad4805daa76a8dcc91a3/50383/tyrion-report-evolution.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </span> <figcaption>An example of a debt report showing the evolution of the debt</figcaption> </figure> <p>This kind of report will help you and your team respond to the question "Are we creating debt or increasing quality?" It is important to know because if the trend of your debt is increasing, you should invest more in quality. Remember you won't go faster by overlooking quality 😉.</p> <h3>Sharing it with the product/business team</h3> <p>If you are reading this, you are probably a developer, and you probably want to spend more time on quality. You may not be able to do so because someone in the business side of your project prefer that you focus on a short term deadline.</p> <p>So you need to negotiate with this person. This tool was also build to help you to do so by providing graph and data to support your case.</p> <div class="blue box"> <aside>💡</aside> If you need a csv file to allow business people have a closer look at the data you can generate it with the option `--csv`. </div> <figure> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <span class="gatsby-resp-image-background-image" style="padding-bottom: 20.54054054054054%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAECAYAAACOXx+WAAAACXBIWXMAAAsSAAALEgHS3X78AAABLElEQVQY0yWP3SuDcRTHnxuFmDLvLSHDmGZr7M3KxRiyWLFZtp5tbGlsj7yMadGKREu4EOWGtJSU0m7kSsqf9fHb41ycU6fP+X7PV3r//KVvchPLwi5Ds1sk87csK0UM0wrm+R2cwUOOrko4AjlMvm3BKAQyRVLHd4LJYPXvqfPh9YtKSZV2fv9GlSGAdlSm0SrjjRXQe1JUG5fRmMN0TazjT53RYImoTGXnWDrAvpilZjhE01iU+pEVbp7K/4Klj28Bh+n1bNBqjzOXOFGdNeYIuvEE7c5V5Ow1Ha41VbzFFhOC+0zHC7QIXi/uaoVw/vIZ6aX8g9Q9j86dYHBGwRXMqe5Sj58Bb5pmWxR57wqt+KLOtEL/VBqjiO1LniJ1+nCH8rQJw/2LRzXyHzOipmVEUM0bAAAAAElFTkSuQmCC&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Sharing quality data with product team is a good way to onboard them on the topic" title="Sharing quality data with product team is a good way to onboard them on the topic" src="/static/64f0ee5fab252f170df44d94c201010c/50383/tyrion-flux.png" srcset="/static/64f0ee5fab252f170df44d94c201010c/1d79a/tyrion-flux.png 185w, /static/64f0ee5fab252f170df44d94c201010c/1efb2/tyrion-flux.png 370w, /static/64f0ee5fab252f170df44d94c201010c/50383/tyrion-flux.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </span> <figcaption>Sharing quality data with product team is a good way to onboard them on the topic</figcaption> </figure> <h2>Conclusion</h2> <p>Keeping a high level of code quality on a project for years is something very difficult. One positive side effect of using Tyrion on a project is that developers can grow their quality awareness while coding. If all developers of your team ask them-selves "Is this piece of code I'm writing/reading the best we can do?" for each feature, you will be on a good track to keep the quality high on the long run.</p> <div class="blue box"> <aside>☝️</aside> <strong>You have a question?</strong> Feel free to either contact me: <ul> <li>📧 by <a href="mailto:maximet@theodo.fr">email</a></li> <li>🐦 on <a href="https://twitter.com/maxthoon">twitter</a></li> <li>🐙 by leaving an issue or a pull request on the <a href="https://github.com/theodo/tyrion">Tyrion repository</a></li> <li>💬 by leaving a comment below</li> </ul> I will be happy to discuss about Tyrion and quality management with you. </div><![CDATA[Write tests for humans]]>/2020/07/write-tests-for-humans//2020/07/write-tests-for-humans/Mon, 07 Sep 2020 00:00:00 GMT<p>Let me tell you the story of the day I almost quit testing my code altogether. For the tenth times this week, the CI failed with this cryptic error message "Received value does not match stored snapshot" and a diff that looked something like:</p> <div class="gatsby-highlight" data-language="text"><pre class="language-text"><code class="language-text">&lt;div - className=&quot;myClassName&quot; &gt; + className=&quot;myNewClassName&quot; &gt; + &lt;div&gt; Hello world + &lt;/div&gt; &lt;/div&gt;</code></pre></div> <p>It felt pointless to fix this test : why does it matter if the class-name is different or I added a div tag? It does not keep me from introducing regressions as nobody in the team reads the snapshots updates.</p> <p>I was so frustrated, I almost broke the computer, slammed the door to quit my job. But instead, I decided to channel this energy into something more positive. I started thinking about how the project tests got into such a state.</p> <p>How come everybody agrees that we need to test our code base yet so many developers hate it? Was I the only one being frustrated? (<em>No I was not</em>)</p> <blockquote class="twitter-tweet"><p lang="en" dir="ltr">When people say they use TDD, I don’t believe them.</p>&mdash; emily freeman (@editingemily) <a href="https://twitter.com/editingemily/status/1298679608489852930?ref_src=twsrc%5Etfw">August 26, 2020</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script> <p>Why is nobody doing something about this?</p> <h2>The Vicious Circle Theory or How you stopped writing test for humans</h2> <p>Here is my theory to make sense of this kind of situation. I have called it <strong>"The vicious circle of testing"</strong> ™:</p> <ul> <li>A Developer changes the code for a feature. <strong>A test fails, but it is not clear why</strong>.</li> <li>The Developer fixes the test. They do not know why the test was here in the first place. They grow used to the idea that it is not worth their time investment to fix it.</li> <li>Next time, this developer writes code, they spend less time and effort writing tests.</li> <li>And back to the top...</li> </ul> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <span class="gatsby-resp-image-background-image" style="padding-bottom: 56.21621621621622%; position: relative; bottom: 0; left: 0; background-image: url('data:image/jpeg;base64,/9j/2wBDABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVGC8aGi9jQjhCY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2P/wgARCAALABQDASIAAhEBAxEB/8QAFwABAQEBAAAAAAAAAAAAAAAAAAMBBf/EABQBAQAAAAAAAAAAAAAAAAAAAAD/2gAMAwEAAhADEAAAAe5O4NH/xAAXEAEAAwAAAAAAAAAAAAAAAAABACAx/9oACAEBAAEFAlhtP//EABQRAQAAAAAAAAAAAAAAAAAAABD/2gAIAQMBAT8BP//EABQRAQAAAAAAAAAAAAAAAAAAABD/2gAIAQIBAT8BP//EABoQAAEFAQAAAAAAAAAAAAAAAAEAAhEgIjH/2gAIAQEABj8CgdWXk1//xAAbEAACAwADAAAAAAAAAAAAAAAAAREhMRBBgf/aAAgBAQABPyHuIYu1j6kKlbkhThnH/9oADAMBAAIAAwAAABAQD//EABQRAQAAAAAAAAAAAAAAAAAAABD/2gAIAQMBAT8QP//EABQRAQAAAAAAAAAAAAAAAAAAABD/2gAIAQIBAT8QP//EABwQAQEAAgIDAAAAAAAAAAAAAAERAFEQMSFBcf/aAAgBAQABPxB+vEClhvPARFVkHw96woFG5nYi7mACABo4/9k='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="Circle of bad tests" title="Circle of bad tests" src="/static/c342ff9c4c4cfdfb238c821c5a87c3b8/07f3a/bad-tests-circle-of-life.jpg" srcset="/static/c342ff9c4c4cfdfb238c821c5a87c3b8/d7fe6/bad-tests-circle-of-life.jpg 185w, /static/c342ff9c4c4cfdfb238c821c5a87c3b8/f4308/bad-tests-circle-of-life.jpg 370w, /static/c342ff9c4c4cfdfb238c821c5a87c3b8/07f3a/bad-tests-circle-of-life.jpg 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </span></p> <p>If this resonates with you, you've been writing tests for the CI to work but not for your teammates to understand your intent. Tests are a collaboration tool, they are meant to be read when they fail by you of your teammates. You need to make sure you can understand them in a month when you have forgotten all about what you were supposed to do.</p> <blockquote> <p>Tests are meant to be read by humans, not your CI.</p> </blockquote> <p>How do you write tests for humans, you ask? Here are my 3 tips for you :</p> <ol> <li><strong>Why</strong>: You need to know why you are testing. Because testing is a universal truth in software engineering, it is easy to forget why you need to do it on your project. Find what motivates you: it will guide you to write good tests that you want other humans to read.</li> <li><strong>What</strong>: You need to focus. I think it is better to do one thing well rather than try to do it all and do it mediocrely. Choose what you want to test, do not try to test it all.</li> <li><strong>How</strong>: Choose the best tools adapted to your goal. They must help you write quality tests that you can read and maintain</li> </ol> <h2>#1: Why do you write tests?</h2> <p>Writing great tests for humans takes time, you need to be invested in what you are doing. You cannot be if you do not know exactly why you need to write great tests.</p> <blockquote> <p>Do not write tests because you have to. Make sure you and your team know why you write tests.</p> </blockquote> <p>Automatic testing is often a given when setting up quality tools for your project. Unlike your linter, tests do not fail every day, they are a longer-term investment to code quality. It is easy to lose sight of why you are doing it.</p> <p>Before starting to set up your test environment you should clarify why you are testing.</p> <p>What are you hoping to gain in the long term? Better documentation? Fewer regressions bugs? Developer experience tool for TDD?</p> <p>Make sure you and your team know why you are doing it. Because it is so easy to fall into the vicious circle of bad testing, if you all know why you are driven to write tests, you can catch early when you are deviating from your goal. Making your goal clear will give your teammates <strong>ownership</strong> over your test and they will help you push for improvements when needed.</p> <h2>#2: Choose what parts need to be tested first</h2> <p>If we chose to use snapshots on my project, it was to go faster. We wanted to test all of the codebase without investing much time. By aiming to keep a 100% coverage, we started writing tests that needed to be maintained every time we added features. They could not help us catch regressions because they changed too often. No matter how small the investment was, it was not worth it.</p> <blockquote> <p>Do not aim for a 100% code-coverage of generic tests, aim for 100% of readable tests</p> </blockquote> <p>You do not need to test everything. Your set up may not be adapted to every aspect of your code.</p> <p>I always write unit tests for very logical functions :</p> <ul> <li>They are hard to read and easy to break. A test is a safeguard to make sure another developer will not break it.</li> <li>It is cheap to set up and to write.</li> <li>It is fast so it does not slow down the ci.</li> </ul> <p>But for most part of the code, it is a case-by-case study.</p> <p>Should I test my frontend app UX? If I am writing a complex single-page app, UX is very likely to evolve. Testing it may be my priority so I can prevent regressions that have a big impact on the user's appreciation of my app. I want to write integration or end-to-end tests for this.</p> <p>Should I test your frontend app UI? If I am writing a shared-component library, I am focusing on UI and have a few logical components. My first mission is to provide my users with reliable and consistent UI. Testing UX logic is a bonus, it is ok to make choices and not to focus on this at first.</p> <h2>#3 Choose your tools right</h2> <p>There are a lot of great testing tools available. Now that you know why you want to write tests and what parts of code you want to focus on, you can find the tools that fit your needs like a glove</p> <p>On another project, I chose to test my frontend React app with react-testing-library. I wanted a tool that :</p> <ul> <li>would test our React app components logic and document what they did</li> <li>would help new developers feel confident when starting on the project</li> <li>would help us focus on these goals even if the teams changed</li> </ul> <p>That is why I chose react-testing-library to write integrations tests on it. It turned out to be a great developer experience. I also had a personal goal of trying this new tool that had a lot of great reviews.</p> <p>Find a tool that makes writing test fun, or that makes you want to do test-driven-development, you will only write tests for humans this way.</p> <h1>You are ready!</h1> <p>Now you know: if you ever find yourself thinking testing is useless or painful to maintain, you are probably in a very vicious circle of testing. If you want to escape this, here are the steps to follow:</p> <ul> <li>Remember tests are a documentation and collaboration tool, they should be made to be maintained by teams and read by humans</li> <li>Make sure your team knows why you are writing tests. This will motivate you to write good tests that you can be proud of.</li> <li>Do not try to do it all. Coverage is just a number, it is not a token of code quality. Focus on testing some parts well rather than all of it in a poor way.</li> <li>Choose the tool that fits you and that makes the developer experience fun</li> </ul><![CDATA[Software-based CPU power consumption using PowerApi]]>/2020/09/power-api-deep-dive//2020/09/power-api-deep-dive/Thu, 03 Sep 2020 00:00:00 GMT<p>The ICT sectors carbon footprint represents now around 4% of the global CO2 emissions. To minimize that, one of our goals at Theodo is to design sustainable web applications with minimal carbon footprint. And in order to know how performant we are at this, we need to be able to evaluate the power consumption of our applications and more specifically the power consumption of the VMs we use in the cloud. Unfortunately, at the moment no cloud provider is able to provide such a data. So we had to look for a solution.</p> <p>Our research led us to <a href="http://powerapi.org/">PowerApi</a>, an open-source project developed by the Spirals research group from University of Lille 1 and Inria in France. It allows people to estimate the CPU’s power consumption induced by an application without any external device like a wattmeter. As we can't put wattmeters to measure the VMs consumption in the cloud, this looks like a good option.</p> <h3>The magic formula</h3> <p>From the <a href="https://powerapi-ng.github.io/powerapi_howitworks.html">documentation</a>, we can read that PowerApi collects with a sensor raw data from the hardware of the server. Then PowerApi applies a “formula” and voilà: we have the power consumption of the monitored software stored in a database.</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 691px; " > <a class="gatsby-resp-image-link" href="/static/4bd8e3d93597aa2ec0781c2728cd1b18/e185b/magic-formula.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 55.13513513513514%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAALCAIAAADwazoUAAAACXBIWXMAAAsSAAALEgHS3X78AAABHElEQVQoz4WSyY6EMAxE+f8v49hnDggJsYZmJyxZZh4YdWsujIksk7jsqjhB3/dxHDdNcxzHz39Gzr7vn8xgWZayLClhjPHeP4PHcZwuu8HruvaXETwjnbdt9+4uu8FaayrN80xgrX1AsmD3hzbIJEnoDJLm27Zx5i9zzol3pzvrApaEr+aiKNq2Ff5pmuKHYcCjsKqqeZ5mPVlvWE2r8jz/0gajL5OqWZYppahNvF6mV73t93WwCRJ/g9EgbKFNDD+hzdnJmc8j2Bl7mMN89mVsAdyYM/XgD5L+NOeXgB3KSZN+6JCGHHqgiITX63Vq5oVwAHOS8FEUqUbRbzebcUbA0ziJWhqSX9d1GIb3nLlzSj48EiElCZLJdH8BM6N/fBMfyz8AAAAASUVORK5CYII='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="The diagram to get the power consumption" title="The diagram to get the power consumption" src="/static/4bd8e3d93597aa2ec0781c2728cd1b18/e185b/magic-formula.png" srcset="/static/4bd8e3d93597aa2ec0781c2728cd1b18/1d79a/magic-formula.png 185w, /static/4bd8e3d93597aa2ec0781c2728cd1b18/1efb2/magic-formula.png 370w, /static/4bd8e3d93597aa2ec0781c2728cd1b18/e185b/magic-formula.png 691w" sizes="(max-width: 691px) 100vw, 691px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <p>After reading this I asked myself two questions : what is this “raw data” and how did they build this magic formula ? Fortunately, we can all have access to their <a href="https://powerapi-ng.github.io/index.html">research papers</a> where they explain everything. So let’s dig a bit deeper.</p> <h3>A Power Model is used to approximate the power consumption</h3> <p>The formula is what is called a Power Model in the academic field. A power model is a model that approximates the power consumption of a hardware based on different factors.</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 630px; " > <a class="gatsby-resp-image-link" href="/static/f5ffac46d789849cf15f6e76afa09f61/f058b/power-graph.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 48.64864864864865%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAKCAIAAAA7N+mxAAAACXBIWXMAAAsSAAALEgHS3X78AAABbklEQVQoz3WSy2+CQBDG/dtNevVs2oOJjdGDJk1N+rBK1KQ+UEuFoAskClQUkAooIGDBjsVXbDuHj53N/L6dnSW2PYRhGLZtB0HwdRabzcb3XZBD6odheERiJ1jXJxMJeMdxIl2vHXmmfIgGz88RQgwaStOZZVlQHFnsYSiFPNLtWUA7cw1Mp+/9PkG88TwPvic4+kiSJAg8mLfbPYKgYceybEVRNE0jSZJjWZZjcBxHiIUDLuHRaAT6/FS5uclglUanQ4BLPH6VTufKL/VisVQqYZlMrlptwM2P3e1hVVWXpglAKnWbz98nEteN1x50yHHc40MNw1r1eiuZTDWbeLCDzwYW8avlkqJo13UFQaTpQTabjyrK5Voue8cwAkUNCoWioqiXA4uS6UzWdWPtOLIMi8Wnpo35seu5CA273S4UwOVFUbyEoxyOBfU8b6Evdl7ShKSo1cqCt4exhz+PYprmH/DvgD/E83yI/wq+AQ//KRRh1rBrAAAAAElFTkSuQmCC'); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="A graph of the estimate of power consumption compared to real consumption" title="A graph of the estimate of power consumption compared to real consumption" src="/static/f5ffac46d789849cf15f6e76afa09f61/f058b/power-graph.png" srcset="/static/f5ffac46d789849cf15f6e76afa09f61/1d79a/power-graph.png 185w, /static/f5ffac46d789849cf15f6e76afa09f61/1efb2/power-graph.png 370w, /static/f5ffac46d789849cf15f6e76afa09f61/f058b/power-graph.png 630w" sizes="(max-width: 630px) 100vw, 630px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <p>I knew that each CPU has a specific TDP on their <a href="https://ark.intel.com/content/www/us/en/ark/products/83356/intel-xeon-processor-e5-2630-v3-20m-cache-2-40-ghz.html">specification page</a>: “<em>Thermal Design Power (TDP) represents the average power, in watts, the processor dissipates when operating at Base Frequency with all cores active under an Intel-defined, high-complexity workload</em>”</p> <p>So when I pictured the formula, I thought it was going to be quite simple and something like:</p> <p>Consumption = Idle power consumption + %CPU load * (TDP - Idle power consumption)</p> <p>Reading their <a href="https://hal.inria.fr/hal-01827132v2/document">research paper</a>, it turns out it’s more complicated: “the CPU load does not accurately reflect the diversity of the CPU activities. In particular, to faithfully capture the power model of a CPU, the types of tasks that are executed by the CPU have to be clearly identified.”</p> <h3>Power consumption is related to the CPU activity which is not exactly the CPU load</h3> <p>The types of tasks in a CPU are also known as events. Each processor type has <a href="https://download.01.org/perfmon/index/snb.html">hundreds of different events</a> with potentially a different power consumption. To have an acute idea of what the CPU is doing and thus knowing the related power consumption, the PowerApi teams decided to rely on those events and count them.</p> <p>To do that they rely on specific counters of the CPU: the HPCs (hardware performance counters) : “We therefore decide to base our power models on hardware performance counters (HPCs) to collect raw, yet accurate, metrics reflecting the types of operations that are truly executed by the CPU.”</p> <h3>Learning phase: creating the Power Model from the regression analysis of real power consumption</h3> <p>The goal of this phase is to approximate the amount of power consumption per HPC events during a workload. It’s made possible by measuring both real Power consumption of the server and counting HPC events in real time.</p> <h4>Step 1: select an unbiased set of workloads</h4> <p>To ensure that the data is not biased to one kind of CPU activity, the team used an academic set of different workloads called <a href="https://parsec.cs.princeton.edu/">PARSEC</a>.</p> <p>“To explore the diversity of activities of a CPU, we consider a set of representative benchmark applications covering the features provided by a CPU. In particular, to promote the reproducibility of our results, we favor freely available and widely used benchmark suites, such as PARSEC.”</p> <h4>Step 2: get the correlation between real power consumption and HPC events</h4> <p>The team measured the real power consumption using a bluetooth power meter : the <a href="https://www.alciom.com/en/our-trades/products/powerspy2/">Power Spy 2</a>.</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 344px; " > <a class="gatsby-resp-image-link" href="/static/c3cb8e929a2d9b773e7bccdc7c450eb9/75d93/power-meter.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 153.5135135135135%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAfCAIAAABoLHqZAAAACXBIWXMAAAsSAAALEgHS3X78AAAFfElEQVQ4y32V+VMTZxjH+Wf6m9NqK1Wp09FaW9uZOtIRxs6IVisMVVTkEKHKJdhSQZAOR4AYspCQZLPZ7JFs7hBICJBwJLvkDjm4chEIFhzKYVdQUBv7zP6wM/t+nu/7nJvy8o1tb2+vra4uL8envW7zqNE4NNivVSpkOI5CIqhPwAcwFAwGpndP7iIpe/Dz5yuBaY/bQYlFvbUVha1//QHy2SjC16ilxiGNaVQ/ZjbKpfDa2uoevw8nEol4PB6NRnRaafaVzLLivMV4eH4hGIrMLYRm/AGv2+s06DUvXqztISl7d04kluPxxUg0opIjZcW5XA5zITzr87vpZ3ra5fbY7Q7K6bRtb20lhROLi4vRWISQQHU1JWaTfj4U9AfcwRlvIOj1eB0kZaEoy9b/wdGIjIAeVRVp1IQ/6DONmeRKFYcPNjS3QmLUbrdtbm4mh2OxaCg0K5NCjyoL+rUqn3+GzQPv1/x577eK/MJSFpvjdrs+CEeiodlZn1wK1VYWKhQYSVJNbR0l5TVFZZWFpZVCocjhsG1u/QembWXleTiyMDcfkEtFNIxj0PjEBNDL6Qa4uAgcUcAmg5aaIre3t5LDofD87JxfTsB0zGIxT2/Q83kghwsqcWRUAQ1oVR+E6SYJhWeDM9MKmfj3qiIhCGj6lQSBSyQIgsIwAiEoRJKWvfZ6T3lpdt4fCHgUMlq5kM9nQ7CAyWznctlAD5OQ4f06HUWRyeF4POLzu3w+Nw3XVhVyOEyJBIPFAlwCSwixXj8wMKCzWidfvmW78CtnsVjYO+3weJ0KAq6pKOgBGGq1UqmSi1EhX9CLoEIxAlmSwa+MrpPLPeVx2+hsV5Xf6QU6BGAfX8Dh9gHcPjaGi7RaDUVR78O7YUSjIYfD6rKTsh0Y6G7HcFgEg1xut0DAgUQ8bb+SopIpv1hb83qcdofVNmWRSaCKB/ndrFa1RoFiMIvdQXtRqnCFCrOSYztBvjXPGxsbHjs1ZZ2w26wUOSHBwfKyWyxmi0xO10lEELBaiek0uFolJknzXo5ew+GFOa9zil4DVnLcYh3DUMH90ludjCc4xsdRPj1kGiVq0Mk0b+D9TbK+vk6Pa8DvdDooy6R5ctKEILyykpsd7fUSjCfFhUo5QssO6+WDernNYXkHXlpadDrJQMDlck1NjJsmxo1iEbf07o2Otsc41kdIhTxOO5NR11hX2t1VPzykeSdh83MzpGXM5bbZbFbz2PCoaVAk7C0pym1rqZMTYHNjdW1lPruzAQaZ3cyn1ffz1Sp0bfXvXf0Uv88zMmqwkhOTk+ZRk37IqBUKgKKCbEbb48aGqpxfMrlAq1qBGAZkPA7j3HcnAHZLaGFuB95KmQn6CQIdHtHTLoaGdLoBFb+PVXjnWmlJXvrZU1cu/SiG2OOmQeOgouVpza2cCxC/y+u2vVb+Z30dFvIEPGCgX0mTaq2Mx2Vez8n64fuT35w6lvdrFsjrlGCgRo2wuprOp5/O/vl8eUnusF65FI+9KtViLMZhs3pYnVqtQq6QcHu6rl7O/Pqroye/PFJ05zoMAv0q7OmTh80NFWXFNxobql1O0jxqQEXg6w6jC2Y2jUilmJRAe7oZV7IyTp44lpZ2OP3st51tTRyAkZuddTvv6oN7+Z2tDeOmYRqh1+h+b68kEhKJGMVF7Gctly6e/+L450fTUo8eOZxz7fKT+prmpkfFBdfvFtwcMRqW4vEdaHsfXl5agiEBgghZXa0XL2akHU89lpb66WefHDz0cVHB7V52l0ohXV1dTbK3d66xqdOq2M8YLc31GRnnDhz46OChA4dTD1U/rAyHQ3saNJzkR7f72et2dTJaf7qQeebM6by8PINev3t0c3ODfnlblrZ/AUo8rpATaDJ0AAAAAElFTkSuQmCC'); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="A power meter" title="A power meter" src="/static/c3cb8e929a2d9b773e7bccdc7c450eb9/75d93/power-meter.png" srcset="/static/c3cb8e929a2d9b773e7bccdc7c450eb9/1d79a/power-meter.png 185w, /static/c3cb8e929a2d9b773e7bccdc7c450eb9/75d93/power-meter.png 344w" sizes="(max-width: 344px) 100vw, 344px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <p>Real time in the context of this experiment is that every 250 ms:</p> <ul> <li>The Power Spy 2 send the power consumption of the last 250 ms to a collector</li> <li>The number of HPC events of the last 250 ms are sent to a collector</li> </ul> <p>By doing so we obtain a lot of data linking power consumption and HPC events.</p> <h4>Step 3: select the HPC events that are the most correlated to power consumption</h4> <p>The two final steps are a mathematical ones. Step 3 is about removing the HPC events that are weakly correlated with power consumption to simplify the step 4.</p> <h4>Step 4: Create the Power Model using ridge regression</h4> <p>At this step, the team uses an advanced mathematical method of regression (<a href="https://en.wikipedia.org/wiki/Tikhonov_regularization">ridge regression</a>), to get the 2 or 3 HPC events that will be able to predict the power consumption of the software on the machine. This is what we call the <strong>Power Model</strong></p> <p>Here is a diagram of the whole phase: <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 670px; " > <a class="gatsby-resp-image-link" href="/static/00426fcd0d2fdc0a23cab104809b244b/d67fd/learning-phase.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 65.4054054054054%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAANCAIAAAAmMtkJAAAACXBIWXMAAAsSAAALEgHS3X78AAACI0lEQVQoz12S227aQBRF+WyUPqYfwB+kP9C8RGpICFIiBCRcbOMb4WJsgsE2NjbmZgIBukxaNepIHp2ZOWfO2nucOR6P6/V6s9ksl0tmlgRfd+I4TpLNYhG/v7+zXCwWSZIs4nSZ+fj48DzXtm3TtEzT5Hg4tFqtlmlZjuOoqqrrWqHwoOu67/ue51Uq5bv7e1lRwjBMi6fTqes4tj32XM913X6/3+10yI7mc7IlSXp6eqrV62R3Op1ms1EqlRRFgSsDJ/VhFFF3Op2SZEuSLMtBEJzJV/P5fDSygeciy7KoAfNTWoaCwPfpNhiYb29vhmEosnxzc1Or1R0XIEeSRFXVREmaTCZB4FcrlWZTUDXV94MMSq6vf7JVbzRAFUQRyKurH4VCgWU2m+11u1iANHC+XVwMDINgZNupZmTkcjnIt9stJMxQ0R/xtKrX0/5wURxFUbvdxm3g0YLeFPvvOPKtV0tN0yBXFBWp3FKtVorFYqVahXG327V1PZ/PG8aAfqlheEMTKrHx++Xldptw92TirNKxjKLw9fUV53neTZLgQq/XC6PwT/FwOETeeDwmg4AzUD+x2RyNRhhJzvxcj1Qekni/3//D5hZm0h+KD4+Pj1I6RGiFplB7qT0/vwiCwBWqIhPDAu//xTQUBeH29le5XEaw3GqJothoNLvdLlpmsxn/H5Wu5/HyXw1LB5aAhFZc5Zg4Pg+s5ogGh8OBNAQz/wajTLXH4LmbfQAAAABJRU5ErkJggg=='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="Diagram of the learning phase" title="Diagram of the learning phase" src="/static/00426fcd0d2fdc0a23cab104809b244b/d67fd/learning-phase.png" srcset="/static/00426fcd0d2fdc0a23cab104809b244b/1d79a/learning-phase.png 185w, /static/00426fcd0d2fdc0a23cab104809b244b/1efb2/learning-phase.png 370w, /static/00426fcd0d2fdc0a23cab104809b244b/d67fd/learning-phase.png 670w" sizes="(max-width: 670px) 100vw, 670px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <h3>The final Power Model includes memory, disks, motherboard and fans</h3> <p>As the Power Spy 2 measures the power consumption of the whole server, all other hardware components (memory, disks, motherboard and fans, ..) are integrated into the power model. It works well because the CPU is the most energy consumming component.</p> <p>The final power model is P = Pidle + Pactive, where Pidle corresponds to the power consumption of the CPU when no program is running and Pactive to the additional power consumption from a workload.</p> <p>For example, one power model for the Intel Xeon W3520 is</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 412px; " > <a class="gatsby-resp-image-link" href="/static/6881b1f29015cb72d200dd124d619542/9e32a/power-model.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 34.5945945945946%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAHCAIAAACHqfpvAAAACXBIWXMAAAsSAAALEgHS3X78AAAA1klEQVQY06VPXXPCIBDk//82a9oxVTtTIk0bYlI+BEmAAInXZHRafew+AHfs7u2h6Ypxusc4Pvb+AC2MqvoihGh9dt5x/r3d7rLnl7IsHy1+l2i5+r5rmqaua2ud1hrenPOu72+8EEJK8S4JWj46Y0DjvYfSO2vMOd9slNacMescNKUQGOP9/s1aC9bL/B/xBzm85vk7xm3bKqXk6QRji+LwOUMrJYQwxsAWGBeU0orSlK7iGAKwGWNSyhhjSml2JFm2dnOWYRjghGhPq9Xx2Nw2R9M/cAFlKpObUvGgCgAAAABJRU5ErkJggg=='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="A power model for Intel Xeon W3520" title="A power model for Intel Xeon W3520" src="/static/6881b1f29015cb72d200dd124d619542/9e32a/power-model.png" srcset="/static/6881b1f29015cb72d200dd124d619542/1d79a/power-model.png 185w, /static/6881b1f29015cb72d200dd124d619542/1efb2/power-model.png 370w, /static/6881b1f29015cb72d200dd124d619542/9e32a/power-model.png 412w" sizes="(max-width: 412px) 100vw, 412px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <p>With e1 and e2 two HPC events.</p> <p>To get a real time power consumption of this server PowerApi has then just to count those events.</p> <h3>Real life application</h3> <p>As Cyrielle has explained in this <a href="https://blog.theodo.com/2020/05/greenit-measure-server-energy-consumption-powerapi/">blog post</a>, we were able to measure the power consumption on our local machine. But we couldn't use it to calculate power consumption on AWS servers which was our original goal. Indeed the research team can't do the learning phase on the AWS servers (yet?) as they don't have access to it. So the PowerAPI software can't get the data needed for its Power Model.</p> <p>Thanks to this project and the things we read to try it, we learnt a lot:</p> <ul> <li>The CPU is the component in a server that consumes the most (~100 Watts). The memory is around 10 Watts for 32GB and a SSD consummes around 2 Watts.</li> <li>CPUs, when idle still consume <a href="https://www.extremetech.com/computing/280364-why-intel-tdp-measurements-dont-reflect-real-world-power-draw">around 1/3 of the peak consumption</a>.</li> <li>The next generation of Intel's CPU may see a <a href="https://www.anandtech.com/show/15841/intel-discloses-lakefield-cpus-specifications-64-execution-units-up-to-30-ghz-7-w">91% reduction in power consumption</a></li> <li>ARM processors have a <a href="https://www.androidauthority.com/arms-secret-recipe-for-power-efficient-processing-409850/">very low energy consumption</a> compared to x86 processor(~2 Watts vs ~100 Watts)</li> <li>AWS provides <a href="https://aws.amazon.com/ec2/graviton/">arm-based EC2 instances</a> if you look for efficient way of reducing the carbon impact of your web application.</li> </ul> <p>The next idea that looks promising to have a rough idea of the power consumption of our application is the <a href="https://codeascraft.com/2020/04/23/cloud-jewels-estimating-kwh-in-the-cloud/">"Cloud Jewels"</a> approach by Etsy adapted for AWS.</p><![CDATA[Prevent AWS from Reading Your Step Functions Data]]>/2020/08/secure-aws-step-functions-sensitive-data//2020/08/secure-aws-step-functions-sensitive-data/Mon, 31 Aug 2020 00:00:00 GMT<p><strong>AWS Step Functions</strong> is the perfect tool to handle long-running, complex or business-critical workflows such as payment or subscription flows. However, a naive implementation could <strong>put sensitive data at risk</strong>.</p> <h2>What is AWS Step Functions?</h2> <p>AWS Step Functions is an <strong>Amazon cloud service</strong> designed to create state machines to orchestrate multiple Lambda functions, branching logic and external services in one place. It presents most of the advantages of the <a href="https://blog.theodo.com/2020/06/go-serverless/">serverless services</a>: the scaling is immediate and the pricing is based only on the computing time and the number of step transitions.</p> <p>The state machines can be defined directly in the AWS Console with a JSON configuration and you directly have a graph representation of your workflow.</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 374px; " > <a class="gatsby-resp-image-link" href="/static/ee0f3dc92acca694428f4444beb73d29/52dba/aws-step-functions-state-diagram.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 119.45945945945947%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAYCAYAAAD6S912AAAACXBIWXMAAAsSAAALEgHS3X78AAAELUlEQVQ4y5VUa0ybVRg+34Bx3bz8wAtbIGbr/EvUZksNRhKWEDVcLFH4g01kGkpZZaNMhwnxlmGVhL/GhEQSQ4g/1D+GKAmhKxUGZFszKZe20AuXlpZCS1su7fF5T9utSDP1JOc7t/f2PO/7fow9ZsQXuiRat+/qyvb+0jVZ/tAW0tk7dU1i/3e4J7RCadXU+aR/uv0Bt9/gO3c/GqQ7HuyULL9rHm9ga2tLrD6fjwUCgayg514OnQPmr8q8U1fDfKmLB2Y7JlgLZ/xgJTsc9GQHg8ETwgHnbG1t7bhRj8eT0VnI9Oal4ExLv+PHrOJ/vpGxjIMio7GxsVGACLv39va+jUajveHw7tehCO+xOQP6wzj/LBQKfrO/v3/r8PCwHxG+lzLqdrsfGYMRhsds2kP47O7uLqcRCoXEjEYiOMV4NBrh29vbHA7FO4KYJZ2VlRW2ubmZJD/dcnJAoRHCH1qtVi24aQW/7+N8BfctCwsLn9tstpvY63BfkzFCeCiKx+NyPNB8GfP8/Pz8K2az+SL25zAvYL4IGsoyOD/OH+BWEwSC4/V6eTgcFlABnxN8uodDgkhixaWlpXmDg4MFFJkkSSLCh5BFFkOhl2DI5HA4DHa73eByuQxOp/M2ppH2S0tLdG8CxF+gfAoquR0dHaKsGhoa/lva6+rq8hUKRX6mt6ampsL6+vrsjAaTRk8j5Mn19XXr6uqqBVm3YLWjLm20xzqHd/fOzs73JN/W1lbU3NwsIqypqRFVotfrjxh8GkoRPHDA5jAs+CI+aY9McySE+/1+A8n39fU9MTAwkA89KYUSPD8yODw8LCExL1AmKcOYz8zNzZ0dGRk5d3Bw8GwsFitGIT83NjZW8K+dMj4+fowCwGtFdrtRTr1IzBfI+g1E8Ak65yaQ6BH9Ldz1goJ3U0aBLKFMTQ3hE3gUnKA8ngc0nhpULukrouWIVuxB0XTyx5IFG1mgKPNvDU5Ui4uL3aj+dgi1IqKrcKJD5Nco+uXlZS3ee5C4qmPKqK+n4PUSHFJXyJGE14xG46tJLkWXRCKRwjTO6E4G+Bcg9zqgK0gPCBSgo5i6REUZpEzCu2h83IlJ0Ampz7f5c7IB6mCIA6KY1E0kAySCAuw/pl68jIv7gDGJBExhnQbBM+iQP3E/4XI5qQY/TfSt/yJkZukNCZuE7Cxk72A1QdYCucaMHFZUVOTJ5fKcI+Vh7TmTdpSqq6tz0Uk5mbqEqdVqJpPJWFVVguOSkhKmVL6dq77yThE7efm0w6D5wTeliZl/U2sZk53sbG88VVtbK4xVVlay8vJyplKpEj9p+oAL4lIYNxgMbHR09IjTdVPbbW7v4m6j5stEEP2iPIaGhhjgMyRV2Ej99R8OpVIpJo033qpnd37VCEX/zPUz0Qe6Brfpeh6df/ruAykll65D429SJplCtZP5WAAAAABJRU5ErkJggg=='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="AWS Step Functions state diagram" title="AWS Step Functions state diagram" src="/static/ee0f3dc92acca694428f4444beb73d29/52dba/aws-step-functions-state-diagram.png" srcset="/static/ee0f3dc92acca694428f4444beb73d29/1d79a/aws-step-functions-state-diagram.png 185w, /static/ee0f3dc92acca694428f4444beb73d29/1efb2/aws-step-functions-state-diagram.png 370w, /static/ee0f3dc92acca694428f4444beb73d29/52dba/aws-step-functions-state-diagram.png 374w" sizes="(max-width: 374px) 100vw, 374px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <div align="center" style="color:grey;">AWS Step Functions state diagram</div> <br> <p>At Theodo, we use <a href="https://blog.theodo.com/2020/04/serverless-framework-partnership/">Serverless</a> as our framework with the Step Functions plugin to write our AWS Lambda functions, define our resources (DynamoDB and Queues) and setup AWS Step Functions and the state machines directly in our codebase.</p> <h2>Example: an identity verification workflow with Step Functions</h2> <p>The previous picture shows an identification verification workflow. It is a piece of a bigger validation flow for banking applications.</p> <p>What it does is request an identification URL to a third-party system, send this link by text message to the applicant and wait for them to complete the identity verification. When it is done, the third-party system calls a webhook, which restarts the state machine and either triggers a retry, sends a rejection email, or continues the global validation process.</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/81235233ce9416bca5bb65a4aa446bcf/50383/identification-workflow.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 75.67567567567568%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAPCAIAAABr+ngCAAAACXBIWXMAAAsSAAALEgHS3X78AAACRUlEQVQoz31Sy27TUBDN37HkU/gDxDewYdkFGwQVQiUCBKGRAKW09KE2rZIW8vLr2tf3/baTVEzjkLQbrCNrZM+ZOXNmWkJ7aQIRrn8T9a+ngz/x5fX0YjApiFQmwN//oLUi25JkP38dHp2dv9z93Du+PDi5SnKqbMXVg2yuXIMNuRISSy1tqKuaf/1xLEwI81vtaiDfB2RrV4VqYXwNYgENueTKSFNZq7Sx1nqhbILoNC5mSRGlGN5xVlJhhFQYE0giVGCqtmSora2fRkhILZRDmEcZjrIyKygEMSJcWigthDLWUypiBGRTSVVKbUGG0n74e8aEAXnglrKhkddgNadzYdl4jJlvYapTlGYF0TY451COtTHG1Sm2sAJptlaBtNEs+/Tt9GIwhro5sS0mXUnykgkw1lfL/nACVaFzyR1gQ4a0xWL5oTd89PTji73TxbzOSruVrYwnhGhttTZceip8Tgz4BIalOZ0khdKhfTh91i539seVd4i4tWHggoRJlL4ZRxADs2BO3lssWF0tbju988dPnu+87c4Xywyb9Z6F1qBNmSrNSXMDMFJBbJKzZk+wuYL5ZIz6Z1eXU7A+FBRk6yAUExJxyRhX8HAhBdM8oxxRhCicGuwJAp7DfmDAkCorN+d5x4e2tg71MslL4Srj5xw0g08rzUzaO/GrL1iYiK3PvrW52ygjoyh//b57NBz1ZylkxFwlBU0QaCYx9Ocq5jLiijzsfIfO95NX7zq77e6bvf32lx7lGsrL1T2sAfE/NJS/j+w8aN5f8DUAAAAASUVORK5CYII='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="Graphical representation of the identification workflow" title="Graphical representation of the identification workflow" src="/static/81235233ce9416bca5bb65a4aa446bcf/50383/identification-workflow.png" srcset="/static/81235233ce9416bca5bb65a4aa446bcf/1d79a/identification-workflow.png 185w, /static/81235233ce9416bca5bb65a4aa446bcf/1efb2/identification-workflow.png 370w, /static/81235233ce9416bca5bb65a4aa446bcf/50383/identification-workflow.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <div align="center" style="color:grey;">The identification flow</div> <br> <p>To perform all of those actions, <strong>sensitive information</strong> will flow through the inputs and outputs of the state machine steps:</p> <ul> <li>👤 Personal information from the applicant which are sent to the third-party service to check his/her identity</li> <li>📱 The phone number of the applicant to send the SMS</li> <li>✉️ The email address of the applicant to send the mail</li> <li>✔️ The result of the identity verification</li> </ul> <h2>Problem: we are exposing sensitive data</h2> <p>With this implementation, we see that sensitive data flow through the inputs and outputs of your state machine directly.</p> <div class="gatsby-highlight" data-language="yaml"><pre class="language-yaml"><code class="language-yaml">state<span class="token punctuation">-</span>machine.yaml <span class="token punctuation">---</span><span class="token punctuation">---</span><span class="token punctuation">---</span><span class="token punctuation">-</span> <span class="token key atrule">SendIdentificationRequest</span><span class="token punctuation">:</span> <span class="token key atrule">Type</span><span class="token punctuation">:</span> Task <span class="token key atrule">Resource</span><span class="token punctuation">:</span> arn<span class="token punctuation">:</span>aws<span class="token punctuation">:</span>states<span class="token punctuation">:</span><span class="token punctuation">:</span><span class="token punctuation">:</span>lambda<span class="token punctuation">:</span>invoke <span class="token key atrule">Parameter</span><span class="token punctuation">:</span> <span class="token key atrule">FunctionName</span><span class="token punctuation">:</span> <span class="token key atrule">Fn::ImportValue</span><span class="token punctuation">:</span> $<span class="token punctuation">{</span>self<span class="token punctuation">:</span>provider.stage<span class="token punctuation">}</span><span class="token punctuation">-</span>sendSMS<span class="token punctuation">-</span>Function<span class="token punctuation">-</span>arn <span class="token key atrule">Payload</span><span class="token punctuation">:</span> <span class="token key atrule">creditApplicationId.$</span><span class="token punctuation">:</span> $.creditApplicationId <span class="token key atrule">creditApplicant.$</span><span class="token punctuation">:</span> $.creditApplicant <span class="token key atrule">phoneNumber.$</span><span class="token punctuation">:</span> $.phoneNumber <span class="token key atrule">message.$</span><span class="token punctuation">:</span> $.smsBody <span class="token key atrule">ResultPath</span><span class="token punctuation">:</span> $.identificationSMS <span class="token key atrule">Next</span><span class="token punctuation">:</span> WaitIdentification</code></pre></div> <div align="center" style="color:grey;">An exemple of sensitive data flowing through our inputs and outputs</div> <br> <p>The problem is: nowadays, most DevOps engineer, developers and even some product managers with access to AWS and can see that information. Furthermore, AWS Lambda dumps are stored in an S3 bucket which is not encrypted with the default configuration. Any malicious access to your AWS stack could <strong>leak sensitive information</strong> far more easily than by corrupting your database for example. The more you spread sensitive information between multiple services, the more you put yourself at risk. Finally, as it is stated in <a href="https://docs.aws.amazon.com/step-functions/latest/dg/data-protection.html">AWS documentation</a>:</p> <blockquote> <p><strong>Any data that you enter into Step Functions or other services might get picked up for inclusion in diagnostic logs.</strong></p> </blockquote> <p>Depending on your data classification or threat model, this could be totally unaceptable. With that in mind, you may completely dump AWS Step Functions, or you may try to find a solution because the tool is still awesome and perfectly suits your needs!</p> <h2>Our solution</h2> <p>To prevent leaking sensitive information, we need to clean every step function input and output. Here are three ways of doing it:</p> <ul> <li><strong>Configure Step Functions to log in CloudWatch Logs instead and set a low expiry date.</strong> This is the easier solution but it still exposes logs for a short duration, and you lose any chance of retrieving old execution informations later.</li> <li><strong>Keep the data in the input and output but encrypt and decrypt it between each step.</strong> This one is a bit more complex but you only need to use an encryption key in your lambdas. However, you are still exposing encrypted data, which can be insufficient for some critical sectors, such as health or defense.</li> <li><strong>Never transmit sensitive data between steps and retrieve it directly during their execution.</strong> This solution requires more services to work and thus more things that could fail. However, it is the more secured and allow permanent recovery of logs.</li> </ul> <p>Second and third solutions also requires break-glass processes to allow developpers to legitimately access log data, for debugging or support matters.</p> <p>Considering everything, we prefered to implement the last solution because we wanted to keep our sensitive data in a single safe spot, where we put a lot of attention to the security. Also, we were already using secured <strong>AWS DynamoDB</strong> encrypted with external KMS keys for the waiting tokens of our lambda functions. Thus, we decided to create a new table to store sensitive data between steps.</p> <p>We configured it like below.</p> <div class="gatsby-highlight" data-language="yaml"><pre class="language-yaml"><code class="language-yaml">resources.yaml <span class="token punctuation">---</span><span class="token punctuation">---</span><span class="token punctuation">---</span><span class="token punctuation">-</span> <span class="token key atrule">StepSentiveInputTable</span><span class="token punctuation">:</span> <span class="token key atrule">Type</span><span class="token punctuation">:</span> AWS<span class="token punctuation">:</span><span class="token punctuation">:</span>DynamoDB<span class="token punctuation">:</span><span class="token punctuation">:</span>Table <span class="token key atrule">DeletetionPolicy</span><span class="token punctuation">:</span> Retain <span class="token key atrule">Properties</span><span class="token punctuation">:</span> <span class="token key atrule">TableName</span><span class="token punctuation">:</span> step<span class="token punctuation">-</span>sensitive<span class="token punctuation">-</span>input<span class="token punctuation">-</span>table <span class="token key atrule">AttributeDefinitions</span><span class="token punctuation">:</span> <span class="token punctuation">-</span> <span class="token key atrule">AttributeName</span><span class="token punctuation">:</span> sensitiveInputId <span class="token key atrule">AttributeType</span><span class="token punctuation">:</span> S <span class="token key atrule">KeySchema</span><span class="token punctuation">:</span> <span class="token punctuation">-</span> <span class="token key atrule">AttributeName</span><span class="token punctuation">:</span> sensitiveInputId <span class="token key atrule">KeyType</span><span class="token punctuation">:</span> HASH <span class="token key atrule">BillingMode</span><span class="token punctuation">:</span> PAY_PER_REQUEST <span class="token key atrule">PointInTimeRecoverySpecification</span><span class="token punctuation">:</span> <span class="token key atrule">PointINTimeRecoveryEnabled</span><span class="token punctuation">:</span> <span class="token boolean important">true</span> <span class="token key atrule">SSESpecification</span><span class="token punctuation">:</span> <span class="token key atrule">SSEEnabled</span><span class="token punctuation">:</span> <span class="token boolean important">true</span> <span class="token key atrule">SSEType</span><span class="token punctuation">:</span> KMS <span class="token key atrule">KeyId</span><span class="token punctuation">:</span> hsm<span class="token punctuation">-</span>key<span class="token punctuation">-</span>id</code></pre></div> <div align="center" style="color:grey;">DynamoDB table definition in Serverless</div> <br> <p>We were already using DynamoDB from our lambdas to save our waiting task tokens and we chose to add another table to save sensitive data between steps. Furthermore, on AWS, you can choose to save the encryption keys of your DynamoDB tables outside of AWS, which was a big plus for us.</p> <p>To avoid repeating ourselves, we coded a <strong>wrapper</strong> that we use on all our lambdas. It does the following things:</p> <p>After the return of the lambda, the wrapper:</p> <ul> <li>Looks for sensitive data keys in the output, remove them and build an object with that information</li> <li>Save the sensitive data objects in a dynamo entry</li> <li>Add the id of the new dynamo entry under a “sensitiveData” key in the output</li> </ul> <p>Before the execution of the main lambda function, the wrapper:</p> <ul> <li>Looks for “sensitiveData” keys in the input</li> <li>Retrieve the content at the given IDs</li> <li>Replaces the "sensitiveData" keys with the resulting object in the input</li> </ul> <div class="gatsby-highlight" data-language="typescript"><pre class="language-typescript"><code class="language-typescript"><span class="token comment">// encryption-wrapper.ts</span> <span class="token keyword">export</span> <span class="token keyword">default</span> <span class="token punctuation">(</span>lambdaHandler<span class="token operator">:</span> LambdaHandler<span class="token punctuation">)</span><span class="token operator">:</span> LambdaHandler <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">const</span> <span class="token function-variable function">decryptHandler</span> <span class="token operator">=</span> <span class="token keyword">async</span> <span class="token punctuation">(</span>event<span class="token operator">:</span> <span class="token builtin">any</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">const</span> decryptedInformation <span class="token operator">=</span> <span class="token keyword">await</span> <span class="token function">getSensitiveData</span><span class="token punctuation">(</span>event<span class="token punctuation">.</span>sensitiveInputId<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">return</span> <span class="token function">lambdaHandler</span><span class="token punctuation">(</span><span class="token punctuation">{</span> <span class="token operator">...</span>event<span class="token punctuation">,</span> <span class="token operator">...</span>decryptedInformation <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">;</span> <span class="token keyword">return</span> <span class="token keyword">async</span> <span class="token punctuation">(</span>event<span class="token operator">:</span> <span class="token builtin">any</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">const</span> clearOutput <span class="token operator">=</span> <span class="token keyword">await</span> <span class="token function">decryptHandler</span><span class="token punctuation">(</span>event<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">const</span> sensitiveInputId <span class="token operator">=</span> <span class="token keyword">await</span> <span class="token function">storeSensitiveData</span><span class="token punctuation">(</span>clearOutput<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">const</span> sensitiveKey <span class="token keyword">of</span> sensitiveInputKeys<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">delete</span> clearOutput<span class="token punctuation">[</span>sensitiveKey<span class="token punctuation">]</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token keyword">return</span> <span class="token punctuation">{</span> sensitiveInputId<span class="token punctuation">,</span> <span class="token operator">...</span>clearOutput <span class="token punctuation">}</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">;</span></code></pre></div> <div align="center" style="color:grey;">Our custom encryption wrapper</div> <br> <p>Last of all, we needed to encrypt the first input at the start of our state machine from the API to finally secure our users information all the way!</p> <h2>Takeways</h2> <ul> <li>Take care about what information you have in your lambdas and state machine events on AWS</li> <li>Always think twice about the data you put in your logs</li> <li>Spread your secrets and security implementation in as few services as possible</li> <li>Use a wrapper if you want to repeat behavior in multiple lambda functions.</li> </ul> <p>Thank you very much for reading my article. Feel free to comment, ask for implementation details, and feel free to share the problems that you encountered with AWS Step Functions.</p><![CDATA[Typescript: use the nullish coalescing operator to prevent bugs]]>/2020/08/use-nullish-coalescing-operator//2020/08/use-nullish-coalescing-operator/Tue, 04 Aug 2020 00:00:00 GMT<p>My goal as a CTO is to improve quality. The score of this game is the number of bugs we find each week. Today I share with you a typical bug that more than one person got caught by.</p> <p>Let's say you want to initialize the audio volume of your react application with the value previously saved in the localStorage or defaulting it to <code>0.5</code> if nothing was saved. You could write something like:</p> <p><strong>Bad example</strong></p> <div class="gatsby-highlight" data-language="text"><pre class="language-text"><code class="language-text">function initializeAudio() { let volume = localStorage.volume || 0.5; // ... }</code></pre></div> <p>The problem is that if the user has saved their volume to <code>0</code> it will be interpreted as falsy and your code will use the default value instead of the right value. Some devs prefer the simplicity of || than putting an explicit if clause. And more than once, they were right because 0 was not a plausible value. But as a standard it is too dangerous. For example someone else could see the code and think the || is a good coding stantard in all situation which will eventually create a bug.</p> <p>Typescript 3.7 comes with the best option: you can now use the nullish coalescing operator to prevent that wrong behavior and safely write something like:</p> <p><strong>Good example</strong></p> <div class="gatsby-highlight" data-language="text"><pre class="language-text"><code class="language-text">function initializeAudio() { let volume = localStorage.volume ?? 0.5; // ... }</code></pre></div> <p>That will be compiled in:</p> <div class="gatsby-highlight" data-language="text"><pre class="language-text"><code class="language-text">function initializeAudio() { var _a; var volume = (_a = localStorage.volume) !== null &amp;&amp; _a !== void 0 ? _a : 0.5; // ... }</code></pre></div> <p>To make sure people from your team use it, you can use the <a href="https://github.com/typescript-eslint/typescript-eslint/blob/master/packages/eslint-plugin/docs/rules/prefer-nullish-coalescing.md">prefer-nullish-coalescing</a> ESLint rule. You can also use it with javascript and this <a href="https://babeljs.io/docs/en/babel-plugin-proposal-nullish-coalescing-operator">babel plugin</a>. If you want to go deeper in the understanding of this operator you can go <a href="https://javascript.info/nullish-coalescing-operator">here</a>.</p> <p>I hope this article will help your team prevent this kind of bugs. For more actionable code quality tips, find me on <a href="https://twitter.com/maxthoon">Twitter</a>.</p><![CDATA[Implement contract testing with Pact to get rid of bugs due to front-back and back-back communication]]>/2020/07/contract-testing//2020/07/contract-testing/Sun, 05 Jul 2020 00:00:00 GMT<style> .citesource, .citesource a { color: lightGray; text-decoration-color: lightGray; } .cost-benefit-alternatives { font-size: 0.84rem; } .tag { border-radius: 0.3rem; padding: 0.1rem 0.3rem; } .high-cost .tag { background-color: rgba(255, 0, 26, 0.2); } .medium-cost .tag { background-color: rgba(245, 93, 0, 0.2); } .low-cost .tag { background-color: rgba(0, 135, 107, 0.2); } .medium-benefit .tag { background-color: rgba(245, 93, 0, 0.2); } .high-benefit .tag { background-color: rgba(0, 135, 107, 0.2); } </style> <p><small class="citesource">Image from the <a href="https://docs.pact.io/how-pact-works#slide-5">Pact documentation</a></small></p> <h2>Exec summary</h2> <p>Modern software development organizations scale their development efforts by spreading the development of a system across different teams. Such projects can have team work issues because interconnected applications that change often, create a lot of errors breaking other teams' application. A typical problem due to a large organization is when the backend team deploys a change that is not compatible with the mobile app and breaks it.</p> <p>Contract testing is a way to make sure that several applications developed by collaborating teams can understand each other. With no need to take extra manual care when modifying code, the teams avoid human errors, get rid of bugs, and on the way, develop faster.</p> <p>With contract testing, a team can easily create a test checking that the application they are developing receives viable responses from the provider application that another team is developing. On their side, the provider application team only has to run a single automatically generated test. This allows both teams to know if something is broken before they deploy their applications: they can talk quickly about this precise problem, with little rework cost and no bug for the users.</p> <p><a href="https://pact.io">Pact</a> is the main framework to implement contract tests, and it works with many languages.</p> <h2>For Whom?</h2> <p>In order to avoid creating bugs coming from miscommunication between two parts of your applications you can set up different tactics, contract testing is one of them. Here are some alternatives with an evaluation of how they perform. You can find some comments on these alternatives in the appendix at the end of the article.</p> <table class="cost-benefit-alternatives"> <thead> <tr> <th>Alternative</th> <th>Maintenance cost</th> <th>Setup cost</th> <th>New interaction cost</th> <th>Learning curve</th> <th>Efficient</th> </tr> </thead> <tbody> <tr> <th>End-to-end testing</th> <td class="high-cost"><span class="tag">$$$</span></td> <td class="high-cost"><span class="tag">$$$</span></td> <td class="medium-cost"><span class="tag">$$</span></td> <td class="medium-cost"><span class="tag">$$</span></td> <td class="medium-benefit"><span class="tag">++</span></td> </tr> <tr> <th>Shared types</th> <td class="low-cost"><span class="tag">$</span></td> <td class="medium-cost"><span class="tag">$$</span></td> <td class="low-cost"><span class="tag">$</span></td> <td class="low-cost"><span class="tag">$</span></td> <td class="medium-benefit"><span class="tag">++</span></td> </tr> <tr> <th>Versioned API without breaking change</th> <td class="high-cost"><span class="tag">$$$</span></td> <td class="medium-cost"><span class="tag">$$</span></td> <td class="low-cost"><span class="tag">$</span></td> <td class="medium-cost"><span class="tag">$$</span></td> <td class="high-benefit"><span class="tag">+++</span></td> </tr> <tr> <th>Massive documentation and conception</th> <td class="high-cost"><span class="tag">$$$</span></td> <td class="low-cost"><span class="tag">$</span></td> <td class="high-cost"><span class="tag">$$$</span></td> <td class="high-cost"><span class="tag">$$$</span></td> <td class="high-benefit"><span class="tag">+++</span></td> </tr> <tr> <th>Contract testing</th> <td class="low-cost"><span class="tag">$</span></td> <td class="medium-cost"><span class="tag">$$</span></td> <td class="low-cost"><span class="tag">$</span></td> <td class="high-cost"><span class="tag">$$$</span></td> <td class="high-benefit"><span class="tag">+++</span></td> </tr> </tbody> </table> <p>All these tactics can be used together where it is possible.</p> <p>In fact contract testing needs some conception to create the contract or update it. But contract testing allows to have an accurate documentation when you want to do this conception. Once you discover a breaking change you still have to take a decision on how to handle it and versioned API are a good way to do it but the contracts will show you what you do not need to maintain because no one uses it anymore.</p> <p>Conclusion: if you develop a distributed system communicating via HTTP APIs with heterogeneous technologies you should find someone to help your team set up contract testing.</p> <h2>Benefits</h2> <h3>Benefit #1: Defects are stopped before wasting others time</h3> <p>Usually to achieve testing of several services interacting, we use some kind of end-to-end testing. This means you can test only after deploying your service to put it on the same environment as other services (e.g. staging, preprod, CI). What is called end-to-end testing here could be called broad stack tests, it matches the definition found <a href="https://martinfowler.com/articles/practical-test-pyramid.html#End-to-endTests">here</a>. If there are several teams, it becomes difficult to have someone responsible for end-to-end tests.</p> <p>With contract testing you can test directly in your continuous integration (CI) workflow which means no one else than the developer invested time in checking the work done, this allows problems to be caught earlier. (cf. process just below)</p> <p>doing > manual local test (few minutes) > push (few seconds) > CI (few minutes) > code review (some minutes) > merge (few seconds) > deploy (few minutes) > manual test on staging > validation by the PO</p> <h3>Benefit #2: Developers know their mistakes and correct them quickly</h3> <p>As you can see on the process above having the result in CI instead of after deployment creates a faster feedback loop for the developer and he can correct it before the merge which means corrections are done faster also.</p> <p>Let me introduce the pyramid of tests briefly, there are several kinds of tests with different drawbacks, the idea is that even if useful, the more drawback you have, the less tests of this kind you do.</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 560px; " > <span class="gatsby-resp-image-background-image" style="padding-bottom: 53.51351351351351%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAALCAIAAADwazoUAAAACXBIWXMAAAsTAAALEwEAmpwYAAACOklEQVQoz2WRy28SURTGu3Gp7rrWuOnKhX+BC+MjbUIVqlJqxcfCxJWLujBpXeDGaBNtUptWAgV8gFEoWC0mgJDKSyjFYgHt8IYyMHdmmAEGgXl4Bxob05OTk3Nuzu/7knMHBEHwrhj8LkulkmsQGBw5noOVbDdmogalbzaCp+DI87zQizK1mwbIVmkzicYHOI4NO22JWDC9HSoj8f5es9sac6tO2u6cdzwYst7+SWbhO9sTjeTDKYD8riYRbGfgn2qdBBSG9uVnE2uH3sjPOFWj7idHTdck7seQ4wVxbbdW6nLd/poIcywLKyQJtASbTKM15Aoed/oH7Z5Bu+uE49vhT45Hv7I9GyFP5jpsu2+570wBlOzBN+LpI67IFFK6mcgrtrO3EgVlsnDMF3tWEi+SIyDc2XfuwySGtnE0VKyderd+2uyTWoOXrAGJ2X/Z7J80B0aM68OvPIUiWakX290DME1WyZ10dcpWGNeUJ/UlubYsU1dlakymBlc1dflyc3SJml4t4LkOdxBu4uiinZZqyQkdcUVD3TXRM6vU9Mfa/RVcoQMwJ/TEmCYTCIvn4v+HKQYvPnxLSrXgugHIl8l7H+rPv9JPnbRqjYByMBU6XPIy5Q10xMPzezCMFk1teT8XrS5aaQQjC7hkER9eAGfnwbl5cOEFLlnCL4rZnPOEs8EUhuANsAfXaeqLbi4TC0NBFsEY0waj/868DjHGDeb9ZssSZczRluVHJ5wXulyj3Wh3/7Cc+Lt/AdExGsgmQ3tjAAAAAElFTkSuQmCC'); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="The pyramid of test with on top the most slow and costly tests and at the bottom fast and cheap tests, from top to bottom: UI, Service, Unit" title="The pyramid of test with on top the most slow and costly tests and at the bottom fast and cheap tests, from top to bottom: UI, Service, Unit" src="/static/201a4647b97f714f8d9642014abf3501/b06ae/test-pyramid.png" srcset="/static/201a4647b97f714f8d9642014abf3501/1d79a/test-pyramid.png 185w, /static/201a4647b97f714f8d9642014abf3501/1efb2/test-pyramid.png 370w, /static/201a4647b97f714f8d9642014abf3501/b06ae/test-pyramid.png 560w" sizes="(max-width: 560px) 100vw, 560px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </span> <small class="citesource">Image from the Martin Fowler article <a href="https://martinfowler.com/bliki/TestPyramid.html">Test Pyramid</a></small></p> <p>The main advantage is that contract tests are lower on the pyramid of tests (between unit for the cost and service for the speed) than end-to-end tests (UI). It means they are faster and cheaper, so we can test more cases which gives quickly more confidence for less money.</p> <h3>Benefit #3: Contract tests are easy to create and maintain</h3> <p>With the pyramid we saw that contract testing is cheap compare to the end-to-end alternative. Our experience is that once set up the cost is close to unit tests (only slower) which are widely used on most projects.</p> <p>Each test, tests exactly one case, this means they do not change often (compared to end-to-end tests where any change can break the test) and that a change is easy to do in the test because the complexity is low (like a unit test).</p> <h3>Benefit #4: Contract tests give auto-documented API usage</h3> <p>Thanks to Pact that registers all your contract tests, you know precisely what are the interactions tested in your project. This is up to date even before going to production and you can easily visualize it from Pact.</p> <p>Services interactions:</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <span class="gatsby-resp-image-background-image" style="padding-bottom: 75.67567567567568%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAPCAIAAABr+ngCAAAACXBIWXMAAAsSAAALEgHS3X78AAABdklEQVQoz4WT63LcMAiF8/7PmWayW18kkBDi5srrdDPruq3GP2xGHxzg+G3724nzW/xx5e37SuzPOAosa9s83H1NuRbo/W6K2z/gHeu83m815VC3okSEiK29i6aZ8sTiEdcwQV9/VukjAxXIbqOwq2pYT6zkUbuWJiN4AY/gSNub5VTc7QiKfN1OhWfgaXSgcYYNkX+8G4ARidpe8KHwCXdRZdxoivSx/Rb/BbupmTi3DnnP9TiH7AMe3XacXdlNIvwFXts8twUVEqeweJLPLGMEN7jlvt7pxtpe4JAWdQ1GbaDmZjoED+XMvBc3a8x1mUd3CnCWvcO0bFoGbB4HPGqWUg7lvXcBUMia89Mu39MubNAsVc2FRDpzH1gl0n2AUpELSF7E9GrPlWWG1kSrRWJLy31UpkqYcew9TfeG+FhBXDtsrGVizS0xf9b6UWtd1xU/k6Ewlkb15O8Lb7uzSI7d2oEITOSkY1f/8fbLT/Vq4/07zvwvkTFyeaNZmfoAAAAASUVORK5CYII='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="A network diagram of applications depending on one another" title="A network diagram of applications depending on one another" src="/static/9ff31afb32d63f7c05a4a46a0058de12/50383/network_diagram.png" srcset="/static/9ff31afb32d63f7c05a4a46a0058de12/1d79a/network_diagram.png 185w, /static/9ff31afb32d63f7c05a4a46a0058de12/1efb2/network_diagram.png 370w, /static/9ff31afb32d63f7c05a4a46a0058de12/50383/network_diagram.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </span> <small class="citesource">Image from the <a href="https://docs.pact.io/pact_broker">Pact documentation</a></small></p> <p>Details of an interaction:</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <span class="gatsby-resp-image-background-image" style="padding-bottom: 71.35135135135134%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAIAAACgpqunAAAACXBIWXMAAAsSAAALEgHS3X78AAABeElEQVQoz41S2XKkMAz0//9f3lIz4YgxPsEHNhg820BmM0kqR5dKJYNaajeQdV2XZZnneXkAjimlUsrtC/A2A+u6lUKEkFX10jT1a9u2TdO0bVXXlFKt9bZtd0qxdjTGOOvQVr1cO0qnaSLWWtYLpQ0XqudCSskYG8cRa7Hg/0LvPTohx4UktefKhbgQNBk7G78om6VxRqtxHDBCHHjkv2kot+0IFDtZjbMYYqcmyg3nXArJ+75nPcQPw3D7HgR6OiZ8iHm9pSUffuXTM+Dr5g9kGFPXNWRqJbVWSkrn4M4OpRQ2cy7Oo3Puk/8E42OM0xTB8eEd3iP2B+DAWBTI8x1wDt+YQODzM60uXei6IOSkdYrxsenMJ+IdGISrkZyX11ZersxxHijzjAWl028AH3YQCMagU3xMMc3pL3gj42JKKtim1R7busKVspW/YDcMvmB/miP+7zkj8lEc+f7NPuHdsOuFPT3VlIkwOlw4aD1pgwI5WveD7H+8bCb5dLOqmAAAAABJRU5ErkJggg=='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="A documentation with json example describing the contract" title="A documentation with json example describing the contract" src="/static/91327e0357823d41d9d968d3593510f6/50383/autogenerated_documentation.png" srcset="/static/91327e0357823d41d9d968d3593510f6/1d79a/autogenerated_documentation.png 185w, /static/91327e0357823d41d9d968d3593510f6/1efb2/autogenerated_documentation.png 370w, /static/91327e0357823d41d9d968d3593510f6/50383/autogenerated_documentation.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </span> <small class="citesource">Image from the <a href="https://docs.pact.io/pact_broker">Pact documentation</a></small></p> <h2>Trade-offs</h2> <h3>Trade-off #1: It is useful only when several teams work together</h3> <p>Tests are used to perform check that are hard to remember and take a lot of time to perform manually. For contract tests this means they are less useful when you only have one or two teams as you usually know exactly what the other team uses.</p> <p>They remain relevant but the cost for an inexperienced team on how to use them could create more problems than it solves at first. For an experienced team the cost is not an issue making them a practical solution even to test the interactions between one frontend and its API developed by the same team.</p> <h3>Trade-off #2: The setup is quite complex at first</h3> <p>To make contract tests work you need several days of work.</p> <ul> <li>All teams need to understand the core concepts and how to write good tests with the good level of testing.</li> <li>You need to add a pact broker in you environment (meaning you have to deal with security issues and keeping it running) this is mitigated by the fact the provided docker container works fine.</li> <li>You need to make sure your providers applications run in your CI. By default there is no need to have a running instance in your CI but this is quite easy to do with docker.</li> <li>You need to find a way to handle state in your provider apps to simulate all kinds of responses for instance you need to simulate your own dependencies to make the test reliable. Depending on the complexity of your dependencies (number of dependencies and number of possible outputs) this can be more or less difficult.</li> </ul> <h2>My Opinion</h2> <p>You can benefit of contract testing in many situations. Once you have overcome the initial cost to set up and understand how you should work with it for a big project it can even be a cost effective solution for smaller projects in your organization. So join the Pact users' community to improve your test stack and help your teams to ship faster. You are not the first, many companies like Atlassian use this tool already as they explain it in <a href="https://www.youtube.com/watch?v=-6x6XBDf9sQ">this video</a>.</p> <h2>Go Further</h2> <p>To know if you can get benefit from Pact you should read <a href="https://docs.pact.io/getting_started/what_is_pact_good_for">this page</a> of the documentation, in case of doubt read <a href="https://docs.pact.io/faq/convinceme">this</a>. Then to learn how to use it you should begin <a href="https://docs.pact.io/getting_started/">here</a> and <a href="https://docs.pact.io/how-pact-works">here</a>. To know where these tests should be used you can read the excellent article <a href="https://martinfowler.com/articles/practical-test-pyramid.html#ContractTests">The Practical Test Pyramid</a> which describes most kinds of test you can use in a project.</p> <hr> <h2>Appendix</h2> <p>If you want more details, here are some elements I used to create the evaluation table of the alternatives in the "For Whom?" part:</p> <p><strong>End-to-end testing</strong></p> <p>Pros:</p> <ul> <li>When the tests is green you are sure everything works together since the tests checks everything at once</li> </ul> <p>Cons:</p> <ul> <li>The cost of maintaining an environment to test everything.</li> <li>Test tend to be very flaky due to external causes.</li> <li>You need to run all tests (and they are slow) before the real deployment of any part, which creates a congestion in your teams workflow.</li> <li>The coverage of error cases costs even more because you need to set up your system to fail in expected ways and because each case is quite long to execute you tend to have few.</li> <li>When you discover a real bug, you still need to work to find the root cause (slow to fix)</li> </ul> <p>Use: When you want to check your most critic happy path and you have a way to have enough control on your system in an environment before prod (set up mocks for external services, set up well known data...)</p> <p><strong>Shared types</strong></p> <p>Pros:</p> <ul> <li>Easy to set up</li> </ul> <p>Cons:</p> <ul> <li>You do not have the vision of what versions work together, this force to deploy the different pieces together which is a smell for down time.</li> </ul> <p>Use: When you have a small enough system that you can deploy all at once and a technical way to share code between consumers and provider.</p> <p><strong>Versioned API without breaking change</strong></p> <p>Pros:</p> <ul> <li>The problem cannot exist</li> </ul> <p>Cons:</p> <ul> <li>Maintenance cost is high, you either have to live with the burden of all your previous choices or find a way to know what is not used anymore (one way is a deprecation plan of older versions).</li> <li>Risk of human error and still create a bug</li> </ul> <p>Use: For public APIs this is a good solution as there is no way to contact each consumer directly.</p> <p><strong>Massive documentation and conception</strong></p> <p>this kind of <a href="https://www.fastcompany.com/28121/they-write-right-stuff">massive</a></p> <p>Pros:</p> <ul> <li>Reduces the amount of bugs of any type.</li> </ul> <p>Cons:</p> <ul> <li>Forces you to make the perfect conception, keep track of every decision, to have comprehensive checklists and to train your whole team to follow all items.</li> <li>Your process needs to be at the highest level, like the NASA for space shuttles.</li> </ul> <p>Use: When lives depends on your software working perfectly</p> <p><strong>Contract testing</strong></p> <p>Pros:</p> <ul> <li>Catch bugs between teams before deployment</li> <li>Easy to maintain</li> <li>Teams can deploy at their own pace</li> <li>Documents expectations of all consumers and ensure they are met with no human intervention</li> <li>Keeps track of what versions are compatible</li> </ul> <p>Cons:</p> <ul> <li>You need to carefully set up your workflow to integrate these tests.</li> <li>You need to create tools to handle complex cases.</li> <li>There is yet another tool to teach to your team.</li> <li>You still need other kinds of tests to check the business rules of each application work as expected</li> </ul> <p>Use: When you want to quickly develop a system with more than two interconnected applications (if you do not control the update of some applications like for mobile, each used versions count). And each application is developed by independent teams which communicate frequently. It can be better to have someone to train your team to reduce the setup cost.</p><![CDATA[Serverless Is More Than a Trend, Best Companies Adopt It]]>/2020/06/go-serverless//2020/06/go-serverless/Mon, 29 Jun 2020 00:00:00 GMT<p>A <a href="https://www.wsj.com/articles/cloud-and-serverless-computing-techs-unsung-heroes-11577109600?mod=article_inline">recent article by the WSJ</a> highlighting the benefits of serverless for organizations points at a building awareness by thought leaders about the impact this new trend can have for companies. It resonates with our own experience, celebrating successful serverless implementations with our clients in the last couple of years.</p> <p>Yet, when pitching prospects about the value they can unlock through this technology, we often find that serverless is not very well known among decision-makers.</p> <p>If the value is clear, why is there a lack of awareness?</p> <h2>The Term Serverless is Confusing</h2> <p>Ben Ellerby defended the serverless concept <a href="https://medium.com/serverless-transformation/in-defence-of-serverless-the-term-764514653ea7">in an article</a>, noting that it “is not a good term, yet it is used to describe a powerful and often misunderstood concept.” So, here is his definition:</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 733px; " > <a class="gatsby-resp-image-link" href="/static/1fed4aa9930227a7cb57b6591fce1829/00b70/serverless-definition.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 63.24324324324324%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAANCAYAAACpUE5eAAAACXBIWXMAAAsSAAALEgHS3X78AAACZElEQVQ4y1VTWXuiUAz1eaZu1bq0VfYqgisKIuCC67Q6+/I2//9fnElC63x9CAmX3JOck5Bbv6Rg81ZTTNczOPM+0vMWo3iCIJ0j3EUSz7chon2MSeJR7gzhfoEp+egQQ3U13CoV3Ok15OLTki5lH4fRCJ2pDX8TYBSN5WJ8SAhwLOBs8THBbONTcV/Ajt9PaFr3qGp3GWD+voCG1UTH66JLYPbMgT4wYI6fxOsDHRbHQwPWpIOqeocPjY+4aebBd4utEipqVcAYNMcBg+2+HHD+8xkXssO3E04/npFetnL28uuM/dcjfv79jTHRV4giF2vZ7Wtn1w45YP6Ko5JOc9JoInQmS080ZZrsx6/nLFFCxlqH+0j0e9chP+pmA07gYvlphdXzWswnnQIaxIJ0Si87bM6p6OdRoQUNxE8D0ZALt3rtK6hQ5pd2T8mmSsnhLiThA5kwTzehQhuaPF/2llMqvKbN2IiurD83dKX81qo+MqnqHOvzhjrLJrogStwFewbjc87xyUfHmJhkue8AOagZddg0YV6fYBsItTUls6YcR2TsWTuhemQZUpGI7wpd7U1DWoPSY5km1sKCKaYhhuEI5tCSQbGprg6NTO/rMgSWh1epbSu4bVcEqKJUxXJF2qWaVkd30oWXTMXcoA+lpwqoMTAJyBBAtrdzXubSQ1kAC80iyo9ZnFMGPeiOBtcfwPZ6MPsmDJdAXANPww40R5fYHBA4fVN7GjqjLh6sxwywVYHld0nHprznnGQFw1Yp2ZLLhUYRNzX6C+qFq8/X86/x/3POKzVLKBOgsxqi8fSAIp39A+s2icNY5iuEAAAAAElFTkSuQmCC'); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="Serverless definition" title="Serverless definition" src="/static/1fed4aa9930227a7cb57b6591fce1829/00b70/serverless-definition.png" srcset="/static/1fed4aa9930227a7cb57b6591fce1829/1d79a/serverless-definition.png 185w, /static/1fed4aa9930227a7cb57b6591fce1829/1efb2/serverless-definition.png 370w, /static/1fed4aa9930227a7cb57b6591fce1829/00b70/serverless-definition.png 733w" sizes="(max-width: 733px) 100vw, 733px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <p>When we first introduce people to serverless, there is this aha! moment when they understand it doesn’t mean there are no servers -- it’s just that they’re further outsourced to cloud providers.</p> <p>This graphic positions serverless in the cloud landscape:</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/815d6aea78bcedf907849d1b9df2ed0e/50383/cloud-concepts.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 65.4054054054054%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAANCAYAAACpUE5eAAAACXBIWXMAAAsSAAALEgHS3X78AAACuUlEQVQ4y2VTW0tUURSeoJeIoKd+QI+BkREWCOFb9FaBPVlQFhEEvhRKmfZgUZGSil2QoGCUFKQwHR+SLowoZaZNouM0c5oZ5+KcOWfO3M9ln3O+1j7qwNiGxV7stde3v2/ttVzYsUzLhhCTEVqXEU4oCMcz+Bsjo537waiESFKhe5Zz37btqt1lMBPcT4kyFn1rULJ5qDpDsaxB1QzwODMtaHSWK5SRJSvTOTfu89g2IDcXT2SmjfmlAB6+eA8hkqhizAH5UjWGfFGtPGQSEI+ZOwErmRYl2nTBYBj2/ETPm6/47os4SZlc0WEYTqQRiCQd0HQmj7iooKzq1YBytoS0VMBv2qfSBSwmszh85glcNa241T1RxTSSkLAqxEm6SjUVEVxPVRRUAEuqAY0sxWwEKRCQSzh2/in21HXgdq8HklKAX0iBPxxJSvgT3SCGOqIbMtbJ/pPM6AWdaiKrJqIjz6DMz+JUyxBqG3txpWME9RcHsPf4XfQNeQk87wBty+fgpZ2SebBU0hGngo9+nERoeQUnmwdx8PRjNLUNo+HySxxo6MKjV5+2Ei2HFc/TqN7/Sa70H7MgiCqWBRFHG3uwv74TF9rcONvyGkeI7Z0+D/qHZ3DjwTtMeVc3/5EA2E7ATK6MDSkPKVvG6FwEX5ZiqGkcwO66LjS1j+HQuT7sqm3H1XujaLj0HPtOdKC158PmRxnUp8SyCpBT5m1gMAadpOj5NLxjgxh/64Z/YRazM3PwTH/DmpCE2+NDt3sOP/yiA6LRQLBtiRyQT4qmGzRqIrWEiJhUhJ6mv55oBqauwZq8DkzfBLydsL33gYV+wD9ErTAOI/QZZmIRlhyk3tVI/iaoi9dB3yquM0amAZZNwMgmwXIJMCUGXQ4jJawgFvAh6l9CPPALpaQfdiYEMxdHPKVApDHk6x/RFaLFH9udKQAAAABJRU5ErkJggg=='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="Cloud paradigms comparison" title="Cloud paradigms comparison" src="/static/815d6aea78bcedf907849d1b9df2ed0e/50383/cloud-concepts.png" srcset="/static/815d6aea78bcedf907849d1b9df2ed0e/1d79a/cloud-concepts.png 185w, /static/815d6aea78bcedf907849d1b9df2ed0e/1efb2/cloud-concepts.png 370w, /static/815d6aea78bcedf907849d1b9df2ed0e/50383/cloud-concepts.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <p>In a nutshell, using serverless gives you the ability to develop your own applications and microservices, to integrate off-the-shelf services like payment or user authentication, and to process events generated by these services or other SaaS. Meanwhile, the size of the engineering team decreases, as the full responsibility of managing the infrastructure is delegated to a cloud provider.</p> <h2>Serverless, Right for Large Companies as Well as Start-Ups</h2> <p>Gartner already mentioned serverless computing as one of the <a href="https://www.gartner.com/smarterwithgartner/top-10-trends-impacting-infrastructure-and-operations-for-2019/">top 10 trends in 2019</a>. Allied Market Research expects the <a href="https://www.alliedmarketresearch.com/serverless-architecture-market">global serverless architecture market</a> to reach almost $22 billion in 2025, from $3 billion in 2017. Led by IT &#x26; Telecom, they expect the revenue to increase in all sectors, with the highest growth in the Media &#x26; Entertainment industry.</p> <p>Large companies drove the market at its beginnings, with use cases such as real-time web applications and data processing; but SMEs will exhibit the highest growth rate between 2018 and 2025, according to the same study.</p> <p>In his article <a href="https://medium.com/@sbrisals/serverless-is-for-everyone-188e3f1b9600sbrisals/serverless-is-for-everyone-188e3f1b9600">Serverless is for Everyone</a>, Sheen Brisals, Senior Engineering Manager at The LEGO Group, explains that “for a majority of start-ups, the tech stack is something they are born with”, therefore “change is easier when compared with a big enterprise”. But “starting small with serverless is perfectly applicable to a start-up as well as to a grown-up.”</p> <p>As cloud providers invest in improving their solutions, more developers adopt serverless for developing microservices and functions, expanding the potential use cases.</p> <p><a href="https://www.theodo.com/en/free-serverless-deck"><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 512px; " > <span class="gatsby-resp-image-background-image" style="padding-bottom: 62.70270270270271%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAANCAIAAAAmMtkJAAAACXBIWXMAAAsSAAALEgHS3X78AAADDUlEQVQoz2P4Txz4ByY/f/358PmH5+8/f/n3/c+f3wzV8041LjzdgITqgWjB6bp5Jzs3XJ104EHb6gtNS890brw69dDD8rknY9v2xHbuSZ2yb8Ph2wwMDlOZnKcxOE1lcAKRjE7TWF1nsHnMZHWZIp25Vrf9kHj8YsGgGQqlm/SmnObzny7gO13Abw6D/ZTsiQcYxILnWeetM8lcbZ69xiJnrWHaKvnIhfLRCxRi5prUbHabdcKwe7dq7Uarvj3x++6HtC81jJ0kF79UIGRh2ZwTDKKB82La9sS07fau2pLcvS+saYdS3EL5uPnKcbPMe/Z4Lj9rN+uQ/6JTjnOOJO245l+7QjZsskrWCsGweWVzjjMIB8wNa9wZULs9rGFHUve+5N59pnkrFJPnaWQvMJ++N2brjaVXPk659jvo8LOAPddVs+bKhfRoF68QiVtYOe8Ig3jwfKDNgXXbEzr26qYuV4idr56+QCd7jm3XKo9NZ+MO3Nz87MfGe/+n3/9dcvll6fYb8avO6daslkxdVLPgMMhmh6INTkUbbfLXaacs08lYrJmzyCBtknd6Y/bGI0EbzyjPOJB2/mXDzR/+e27b7bllsP2GYdVi+awFtUuOMAj5zzXPWmOSvsY0a7V9yTq7slW6RYs1E/plHCsc2tdqzDvD2biNIXWe4MRD+nOOmvYvNe9cYta4VDF/Yf1SsGbHog2uxRu1EpdppizWyVqoX7xINaFXKaherWG5RN9R9oLVDPHz+Op3KUw8aNc7z3HSUvPW5Sr5cxuWgp0tG75QMnShVMRCuYRF6pmL9IoXhs7aW77+Qvz8jXot8+Wrl4h3b1LrWOUwbYn3wuUOU1bYdCxTK13UuOIoA6PLDC7feZw+c7mCFgmGzZVImCeXNktz1QXLIy+MZmxwmzvHb9mc0MWTA9fOc1u8wGrKAsOuhTq1C8Qy51QuPsQQ3LA9qm1PRNveiPY9MV17EibuTp+2K2nh/rgNx4vX76nctq9x566WPbsa9uyp27qzYsPuwtX7clYcSpy8fcn+awADopvk7oC7xgAAAABJRU5ErkJggg=='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="Serverless business case cover" title="Serverless business case cover" src="/static/74befa5a41c139d924b26177829f524c/01e7c/serverless-business-case-cover.png" srcset="/static/74befa5a41c139d924b26177829f524c/1d79a/serverless-business-case-cover.png 185w, /static/74befa5a41c139d924b26177829f524c/1efb2/serverless-business-case-cover.png 370w, /static/74befa5a41c139d924b26177829f524c/01e7c/serverless-business-case-cover.png 512w" sizes="(max-width: 512px) 100vw, 512px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </span></a></p> <p>Download the free presentation <a href="https://www.theodo.com/en/free-serverless-deck">here</a></p> <h2>It’s Not Just About Saving Costs. Serverless Computing Solves Business Problems and Delivers Better Customer Experiences.</h2> <p>Reduced operational and development costs are the most important benefits sought by organizations going serverless, together with the automatic scaling of servers according to demand. On the product side, the increase in developer productivity and reduced engineering lead-time has a positive impact on the speed of delivery, thus on the time-to-market.</p> <p>To overcome the challenges of dealing with legacy applications built during their 100+-year existence, companies like Liberty Mutual embraced serverless computing to solve real customers’ problems. Among the solutions that the IT team have built, Gillian Armstrong (Solutions Architect at Liberty Information Technologies) and Mark McCann (Architect) mention:</p> <ul> <li>A virtual agent for their call center to answer the most common questions.</li> <li>A virtual assistant to help the clients manage their policy documents.</li> </ul> <p><a href="https://www.serverlesschats.com/15/">According to Armstrong</a>, with serverless:</p> <blockquote> <p>The cost is so low, because it's all managed services, and they're able to bring it out, trial it with users for a little bit, make sure the whole system works, and then just scale seamlessly up.</p> </blockquote> <p>At Theodo, we're also obsessed with business outcomes, and we train our engineers to focus on delivering value for our clients' end-users, in line with our Lean culture. When we start a new project, we recommend our clients use serverless components when we're confident it will add a competitive edge to the solution.</p> <p>Check in this video to see how our ground transportation client used serverless for managing omnichannel, real-time notifications to passengers:</p> <iframe width="100%" height="500" src="https://www.youtube.com/embed/l53YU3D-exM" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe> <h2>Serverless Computing Is as Sophisticated as It Is Powerful: the Main Roadblock is Training</h2> <p>According to the <a href="https://www.oreilly.com/radar/oreilly-serverless-survey-2019-concerns-what-works-and-what-to-expect/">O’Reilly Serverless Survey 2019</a>, with a panel of 1500 respondents from a wide range of locations, companies, and industries, some of the biggest challenges are linked to the education and hiring of staff.</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/0b42db1739b9966bd085779d69588bc6/50383/serverless-challenges.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 17.837837837837835%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAECAYAAACOXx+WAAAACXBIWXMAAAsSAAALEgHS3X78AAABCUlEQVQY04WPS0rDYACEcxBXnsK7iCfQE/QIrly6UClCBUEQV0G6MCBtqa2VWmrb/Hk2j2qfMU1jEko/f0/gLOdjZhjl5c2m03Opt02aHYuRHjASId0Pn8arRbvr4oVLbG/OJsn5T8oqWrNOfiiKLVGcUrpqUrqoMV9ExOsNy1VMlsuiHbjRlPLgmbo7hO1O+hm5ZHlRSL7lUh2gCCGYfH6xK1LO7jrsHd5wcHyL608IAx/TNPH9gNlszmnjgRPtmvPGI+l3gjANLNshiVeo9SH7RxWUkW4gDBvH9bjX+lSqfaotmyzLGXu+vG9iOx7G2EMTPWrOAGc5Jd2k6JL9ZQM5+NTSKavv/AJG4R2YM+sMtgAAAABJRU5ErkJggg=='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="Survey results: 43% think training existing staff is a challenge when adopting serverless" title="Survey results: 43% think training existing staff is a challenge when adopting serverless" src="/static/0b42db1739b9966bd085779d69588bc6/50383/serverless-challenges.png" srcset="/static/0b42db1739b9966bd085779d69588bc6/1d79a/serverless-challenges.png 185w, /static/0b42db1739b9966bd085779d69588bc6/1efb2/serverless-challenges.png 370w, /static/0b42db1739b9966bd085779d69588bc6/50383/serverless-challenges.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <p>In the organizations that have not (yet) adopted serverless, “fear of the unknown” comes in second after security concerns.</p> <p>Some components proposed by Cloud providers, like DynamoDB from AWS, have huge potential, but using them correctly is still a skill that requires mastery.</p> <p>To tackle these concerns, Theodo developed <a href="https://youtu.be/mEGZeV0euWs?t=604">EventBridge Storming workshops</a>, adapted from the Event Storming workshop technique by Alberto Brandolini. During these sessions, we work with business stakeholders to define a common language around application fields (e.g. order, payment, and shipping) that we can use to communicate with the team. We then use this vocabulary to design an architecture based on events grouped by domain and well-defined system boundaries.</p> <p>Cloud providers and the user community also recognize the challenges faced in upskilling teams to handle serverless. The good news is that they are doing an outstanding job at <a href="https://medium.com/serverless-transformation">improving the tools</a>, developing <a href="https://www.jeremydaly.com/securing-serverless-a-newbies-guide/">best practices around security</a> concerns, and sharing knowledge on novel ways to solve issues. Theodo is proud to have contributed to multiple frameworks and libraries in the space, an achievement recognized with a <a href="https://twitter.com/theodo_uk/status/1275121519698628608">partnership with AWS</a> and the <a href="https://blog.theodo.com/2020/04/serverless-framework-partnership/">Serverless Framework</a>.</p> <h2>Conclusion</h2> <p>Reductions in Total Cost of Ownership and Time-to-Market are two compelling reasons to consider serverless for IT leaders. Besides, the granular nature of FaaS and managed services allow companies of all sizes to progressively build new applications and replace pieces of legacy architecture without requiring an engineering big-bang.</p> <p>Chances are that you're already using serverless. If your technical stack includes a cloud provider such as AWS, Azure, or Google Cloud, your engineers have probably used their FaaS offerings for data processing or queue services for notifications. You're not starting from scratch!</p> <p>This means that serverless is one of the easiest technologies to adopt. Even if your main infrastructure is still bare metal, you can experiment and iterate with serverless on a limited scope quickly and seamlessly. It will give you a better idea of the benefits and challenges of serverless for your company, right now.</p> <p>--</p> <p>Need help? Contact me at <a href="mailto:alexdb@theodo.com">alexdb@theodo.com</a></p> <p><span>Cover photo by <a href="https://unsplash.com/@dsmacinnes?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Danielle MacInnes</a> on <a href="/@dsmacinnes?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></span></p><![CDATA[Build an Animated Accordion List in React Native]]>/2020/06/build-accordion-list-react-native//2020/06/build-accordion-list-react-native/Sat, 27 Jun 2020 00:00:00 GMT<style> code { font-family: "SFMono-Regular", Consolas, "Liberation Mono", Menlo, Courier, monospace; line-height: normal; background: rgba(135,131,120,0.15); color: #EB5757; border-radius: 3px; padding: 0.2em 0.4em; font-size: 0.7rem; } </style> <p>Accordion lists (or <code>ExpandableListViews</code>) are now a material design staple and show up in quite a few apps. They are a great way to dynamically display information to a user.</p> <p>When it comes to implementing one in your own app, sure, there are libraries that you can use. But implementing one yourself gives you greater control over styling, a smaller app bundle size, and it’s so easy it might even be faster! It’s also a great opportunity to sharpen your skills using React Native Hooks and the Animation API.</p> <p>This is what we’ll be creating:</p> <p align="center"> <img alt="Accordion List Final Result" src="/7c2c141bc5e25e1d8f444382144ced0e/accordion-list-final.gif" style="margin-top:30px;margin-bottom:20px;width:60%"> </p> <h2>Just show me the code!</h2> <p>Alright, alright! Here’s a <a href="https://snack.expo.io/@bgpiggin/accordion-list">snack</a>. Consider sticking around though - you might learn something.</p> <h2>Let’s get started</h2> <p>First off we need a functional component. This will take two arguments or props:</p> <ol> <li>A <code>string</code>. The title in the top bar of the list item.</li> <li>A <code>ReactNode</code>. The component in the expandable section.</li> </ol> <p>For the second of these, React allows us to access child components via <code>this.props.children</code> in class components, or we can specify children as an argument to our functional component. Magic!</p> <p>So our functional component has the form:</p> <div class="gatsby-highlight" data-language="javascript"><pre class="language-javascript"><code class="language-javascript"><span class="token keyword">const</span> <span class="token function-variable function">AccordionListItem</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token parameter"><span class="token punctuation">{</span> title<span class="token punctuation">,</span> children <span class="token punctuation">}</span></span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token operator">...</span> <span class="token punctuation">}</span><span class="token punctuation">;</span> <span class="token keyword">export</span> <span class="token keyword">default</span> AccordionListItem<span class="token punctuation">;</span></code></pre></div> <br /> <p>And we can use it like so:</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">AccordionListItem</span></span> <span class="token attr-name">title</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span><span class="token string">"List Item"</span><span class="token punctuation">}</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">Text</span></span><span class="token punctuation">></span></span><span class="token plain-text">Some body text!</span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span><span class="token class-name">Text</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span><span class="token class-name">AccordionListItem</span></span><span class="token punctuation">></span></span></code></pre></div> <br /> <p>The top bar of our list item will need to be wrapped in a <code>TouchableWithoutFeedback</code> since we want users to be able to open and close the list item by pressing it. This can contain the title we passed in via props.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">TouchableWithoutFeedback</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">View</span></span> <span class="token attr-name">style</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>styles<span class="token punctuation">.</span>titleBackground<span class="token punctuation">}</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">Text</span></span><span class="token punctuation">></span></span><span class="token punctuation">{</span>title<span class="token punctuation">}</span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span><span class="token class-name">Text</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span><span class="token class-name">View</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span><span class="token class-name">TouchableWithoutFeedback</span></span><span class="token punctuation">></span></span></code></pre></div> <br /> <p>Now let’s add a <code>View</code> for the list item body. This is the section that slides in and out when the header is clicked, and will contain our child component.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">View</span></span> <span class="token attr-name">style</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>styles<span class="token punctuation">.</span>bodyBackground<span class="token punctuation">}</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token punctuation">{</span>children<span class="token punctuation">}</span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span><span class="token class-name">View</span></span><span class="token punctuation">></span></span></code></pre></div> <br /> <p>Let’s see what we have so far:</p> <p align="center"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 647px; "> <a class="gatsby-resp-image-link" href="/static/3a3cc689183034ad206f2ff8ce3a7aeb/63577/list-item-static.jpg" style="display: block" target="_blank" rel="noopener"> <span class="gatsby-resp-image-background-image" style="padding-bottom: 36.75675675675675%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/jpeg;base64,/9j/2wBDABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVGC8aGi9jQjhCY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2P/wgARCAAHABQDASIAAhEBAxEB/8QAFgABAQEAAAAAAAAAAAAAAAAAAAIF/8QAFAEBAAAAAAAAAAAAAAAAAAAAAP/aAAwDAQACEAMQAAAB3KAD/8QAFRABAQAAAAAAAAAAAAAAAAAAECH/2gAIAQEAAQUCr//EABQRAQAAAAAAAAAAAAAAAAAAABD/2gAIAQMBAT8BP//EABQRAQAAAAAAAAAAAAAAAAAAABD/2gAIAQIBAT8BP//EABQQAQAAAAAAAAAAAAAAAAAAABD/2gAIAQEABj8Cf//EABgQAAMBAQAAAAAAAAAAAAAAAAAhMQGR/9oACAEBAAE/IbmdGM//2gAMAwEAAgADAAAAEPPP/8QAFBEBAAAAAAAAAAAAAAAAAAAAEP/aAAgBAwEBPxA//8QAFBEBAAAAAAAAAAAAAAAAAAAAEP/aAAgBAgEBPxA//8QAGBABAAMBAAAAAAAAAAAAAAAAAQARcVH/2gAIAQEAAT8QoJEvqmSZJ//Z&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Accordion List No Animation" title="Accordion List No Animation" src="/static/3a3cc689183034ad206f2ff8ce3a7aeb/63577/list-item-static.jpg" srcset="/static/3a3cc689183034ad206f2ff8ce3a7aeb/d7fe6/list-item-static.jpg 185w, /static/3a3cc689183034ad206f2ff8ce3a7aeb/f4308/list-item-static.jpg 370w, /static/3a3cc689183034ad206f2ff8ce3a7aeb/63577/list-item-static.jpg 647w" sizes="(max-width: 647px) 100vw, 647px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </a> </span> </p> <p>Nothing revolutionary right? Bear with me, here’s where it gets interesting!</p> <h2>Now I’m hooked</h2> <p>Next up we’ll want to add a few React hooks to keep track of the state of our component. Here they are:</p> <ol> <li>A <code>state</code> to keep track of whether the item is open or closed.</li> <li>An <code>Animated.Value</code> to animate the opening and closing of the body section.</li> <li>The height of the body section - this is what we will be changing in order to show and hide it.</li> </ol> <p>Or in code form:</p> <div class="gatsby-highlight" data-language="javascript"><pre class="language-javascript"><code class="language-javascript"><span class="token keyword">const</span> <span class="token punctuation">[</span>open<span class="token punctuation">,</span> setOpen<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token function">useState</span><span class="token punctuation">(</span><span class="token boolean">false</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">const</span> animatedController <span class="token operator">=</span> <span class="token function">useRef</span><span class="token punctuation">(</span><span class="token keyword">new</span> <span class="token class-name">Animated<span class="token punctuation">.</span>Value</span><span class="token punctuation">(</span><span class="token number">0</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">.</span>current<span class="token punctuation">;</span> <span class="token keyword">const</span> <span class="token punctuation">[</span>bodySectionHeight<span class="token punctuation">,</span> setBodySectionHeight<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token function">useState</span><span class="token punctuation">(</span><span class="token number">0</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre></div> <br /> <p>Note that we’re using <code>useRef</code> to create the animation object. Since our open state will persist across component refreshes, we want our animated value to do the same, otherwise they’ll get out of sync! <code>useRef</code> acts as a sort of state container for our <code>Animated.Value</code>, elevating it out of the component lifecycle.</p> <h2>Accordion time!</h2> <p>Now we need to tie our <code>Animated.Value</code> to the height of the body view. Our animated value can vary over whatever range we want but it’s easiest to pick a simple range like [0, 1]. Here, 0 maps to the list item being closed and 1 to it being open. As the animated value changes from 0 -> 1 we want the height of our view to go from 0 -> <code>bodySectionHeight</code>. To achieve this we can use the <code>interpolate</code> function:</p> <div class="gatsby-highlight" data-language="javascript"><pre class="language-javascript"><code class="language-javascript"><span class="token keyword">const</span> bodyHeight <span class="token operator">=</span> animatedController<span class="token punctuation">.</span><span class="token function">interpolate</span><span class="token punctuation">(</span><span class="token punctuation">{</span> inputRange<span class="token operator">:</span> <span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">,</span> outputRange<span class="token operator">:</span> <span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">,</span> size<span class="token punctuation">.</span>height<span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre></div> <br /> <p>Which simply maps the <code>inputRange</code> to the <code>outputRange</code> exactly as we described. When we change the value of the <code>animatedController</code>, the <code>bodyHeight</code> variable will scale this value relative to the height of our container. With all this set up, we can now write our <code>toggleListItem()</code> function, which will be called when we click on the item header.</p> <p>We can think about what this function has to do as follows:</p> <ol> <li>If the list item is open:<br> a. Change the animated value from 1 to 0.<br> b. Set the state to closed.</li> <li>If the list item is closed:<br> a. Change the animated value from 0 to 1.<br> b. Set the state to open.</li> </ol> <div class="gatsby-highlight" data-language="javascript"><pre class="language-javascript"><code class="language-javascript"><span class="token keyword">const</span> <span class="token function-variable function">toggleListItem</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>open<span class="token punctuation">)</span> <span class="token punctuation">{</span> Animated<span class="token punctuation">.</span><span class="token function">timing</span><span class="token punctuation">(</span>animatedController<span class="token punctuation">,</span> <span class="token punctuation">{</span> duration<span class="token operator">:</span> <span class="token number">300</span><span class="token punctuation">,</span> toValue<span class="token operator">:</span> <span class="token number">0</span><span class="token punctuation">,</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">start</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token punctuation">{</span> Animated<span class="token punctuation">.</span><span class="token function">timing</span><span class="token punctuation">(</span>animatedController<span class="token punctuation">,</span> <span class="token punctuation">{</span> duration<span class="token operator">:</span> <span class="token number">300</span><span class="token punctuation">,</span> toValue<span class="token operator">:</span> <span class="token number">1</span><span class="token punctuation">,</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">start</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token function">setOpen</span><span class="token punctuation">(</span><span class="token operator">!</span>open<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">;</span></code></pre></div> <br /> <p>I’ve set the duration to 300ms, but you can play around to find your preferred speed.</p> <p>We’re getting pretty close. We’ll need to make our <code>View</code> an <code>Animated.View</code> in order to use React Native’s animation system, and then set the height of the animated view to be the <code>bodyHeight</code> variable.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">Animated.View</span></span> <span class="token attr-name">style</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span><span class="token punctuation">[</span>styles<span class="token punctuation">.</span>bodyBackground<span class="token punctuation">,</span> <span class="token punctuation">{</span> height<span class="token operator">:</span> bodyHeight <span class="token punctuation">}</span><span class="token punctuation">]</span><span class="token punctuation">}</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token punctuation">{</span>children<span class="token punctuation">}</span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span><span class="token class-name">Animated.View</span></span><span class="token punctuation">></span></span></code></pre></div> <br /> <p>Now for a sneaky step. We need to set an initial value for the height of our <code>bodyView</code> so we know what value to return when we open the list item. But, we can’t use the height of the animated view. This is because we are going to set the <code>bodyHeight</code> using the <code>onLayout</code> prop of <code>View</code>. For an animated view this is called many times while the view is animating! This is no good for us, we just want to set it once.</p> <p>To fix this we can nest another regular view in our animated view which can have a constant height. We can then use this view’s height for the value of <code>bodyHeight</code>.</p> <p><code>onLayout</code> takes a function with an event parameter. We access the components height via <code>event.nativeEvent.layout.height</code>. So, putting it all together:</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">Animated.View</span></span> <span class="token attr-name">style</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span><span class="token punctuation">[</span>styles<span class="token punctuation">.</span>bodyBackground<span class="token punctuation">,</span> <span class="token punctuation">{</span> height<span class="token operator">:</span> bodyHeight <span class="token punctuation">}</span><span class="token punctuation">]</span><span class="token punctuation">}</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">View</span></span> <span class="token attr-name">onLayout</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>setSize<span class="token punctuation">}</span></span> <span class="token attr-name">style</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>styles<span class="token punctuation">.</span>bodyContainer<span class="token punctuation">}</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token punctuation">{</span>children<span class="token punctuation">}</span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span><span class="token class-name">View</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span><span class="token class-name">Animated.View</span></span><span class="token punctuation">></span></span></code></pre></div> <br /> <p>Finally we need to set the <code>overflow: “hidden”</code> property on the <code>bodyBackground</code> style. This tells the children of the animated view what to do if they overflow the views box, which is exactly what will happen when the list item is closed. Setting this to hidden means they won’t be visible, perfect!</p> <p align="center"> <img alt="Pretty good!" src="/aa0d7b2856303464341a7daa40bc52b8/list-item-almost-there-1.gif" style="margin-top:30px;margin-bottom:20px;width:60%"> </p> <h2>Realistic slide</h2> <p>What we have is nice, but it would be great if we could give the impression that the body of the list item is unfolding from the title bar. Fortunately there’s an easy way to achieve this! We can add the following properties to the <code>bodyContainer</code> style:</p> <div class="gatsby-highlight" data-language="javascript"><pre class="language-javascript"><code class="language-javascript">position<span class="token operator">:</span> <span class="token string">"absolute"</span><span class="token punctuation">;</span> bottom<span class="token operator">:</span> <span class="token number">0</span><span class="token punctuation">;</span></code></pre></div> <br /> <p>This ensures that the bottom of the inner view is always aligned with the bottom of the outer view, meaning the inner view should slide nicely up and down as the list item opens and closes.</p> <h2>Spinning arrows</h2> <p>Another nice feature would be to have an arrow indicator that rotates as the list item opens and closes. With our new knowledge of React Native animations, this should be easy! As our animated controller varies from 0 -> 1, we want the angle of our arrow to vary from 0 -> π radians or 0 -> 180° for regular folks. Using the same animated controller means it will always be in sync with the opening and closing. So we have:</p> <div class="gatsby-highlight" data-language="javascript"><pre class="language-javascript"><code class="language-javascript"><span class="token keyword">const</span> arrowAngle <span class="token operator">=</span> animatedController<span class="token punctuation">.</span><span class="token function">interpolate</span><span class="token punctuation">(</span><span class="token punctuation">{</span> inputRange<span class="token operator">:</span> <span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">,</span> outputRange<span class="token operator">:</span> <span class="token punctuation">[</span><span class="token string">"0rad"</span><span class="token punctuation">,</span> <span class="token template-string"><span class="token template-punctuation string">`</span><span class="token interpolation"><span class="token interpolation-punctuation punctuation">${</span>Math<span class="token punctuation">.</span><span class="token constant">PI</span><span class="token interpolation-punctuation punctuation">}</span></span><span class="token string">rad</span><span class="token template-punctuation string">`</span></span><span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre></div> <br /> <p>Then we can add our chosen arrow icon, within an <code>Animated.View</code>, into our title container view:</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">View</span></span> <span class="token attr-name">style</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>styles<span class="token punctuation">.</span>titleBackground<span class="token punctuation">}</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">View</span></span> <span class="token attr-name">style</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>styles<span class="token punctuation">.</span>titleContainer<span class="token punctuation">}</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">Text</span></span> <span class="token attr-name">style</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>styles<span class="token punctuation">.</span>titleText<span class="token punctuation">}</span></span><span class="token punctuation">></span></span><span class="token punctuation">{</span>title<span class="token punctuation">}</span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span><span class="token class-name">Text</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">Animated.View</span></span> <span class="token attr-name">style</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span><span class="token punctuation">{</span> transform<span class="token operator">:</span> <span class="token punctuation">[</span><span class="token punctuation">{</span> rotateZ<span class="token operator">:</span> arrowScale <span class="token punctuation">}</span><span class="token punctuation">]</span> <span class="token punctuation">}</span><span class="token punctuation">}</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">DownArrowSVG</span></span> <span class="token attr-name">width</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span><span class="token number">20</span><span class="token punctuation">}</span></span> <span class="token attr-name">height</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span><span class="token number">28</span><span class="token punctuation">}</span></span> <span class="token punctuation">/></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span><span class="token class-name">Animated.View</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span><span class="token class-name">View</span></span><span class="token punctuation">></span></span></code></pre></div> <br /> <p>Here we have used the <code>transform</code> CSS property to pass in a z-angle of rotation for the view (in other words, the angle perpendicular to the z-axis - I had to work this out by trying <code>rotateY</code> and <code>rotateX</code> first!).</p> <h2>Take it Easing</h2> <p>One for the true perfectionists out there. React Native allows us to change the velocity of the animation via the <code>easing</code> parameter. This is a subtle change in our case, but it’s a good habit to get into as it can make a big difference in feel for some animations.</p> <p>Most of the time I use the standard Material Design easing:</p> <div class="gatsby-highlight" data-language="javascript"><pre class="language-javascript"><code class="language-javascript">easing<span class="token operator">:</span> Easing<span class="token punctuation">.</span><span class="token function">bezier</span><span class="token punctuation">(</span><span class="token number">0.4</span><span class="token punctuation">,</span> <span class="token number">0.0</span><span class="token punctuation">,</span> <span class="token number">0.2</span><span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre></div> <br /> <p>Putting this in both our <code>Animated.timing</code> configs, we get our finished product!</p> <p align="center"> <img alt="All done!" src="/7c2c141bc5e25e1d8f444382144ced0e/accordion-list-final.gif" style="margin-top:30px;margin-bottom:20px;width:60%"> </p> <p>Even after all that work our list item is less than 1kb in size once gzipped! I challenge you to find a library that’s smaller. We also have complete control over every aspect of our styling. Stick this in a <code>FlatList</code> component to create a nice scrollable list, or you can just stack them like regular components if you only need a few. Have fun with it!</p><![CDATA[Why it’s time to consider Azure to host your static application]]>/2020/06/consider-azure-static-app//2020/06/consider-azure-static-app/Mon, 15 Jun 2020 00:00:00 GMT<p>The line between static &#x26; dynamic web applications is not what it used to be. With new frameworks like Gatsby or NextJS, the Jamstack is a strong option to get your application running in no time, whilst being able to scale it as needed.</p> <p>When it comes to hosting your app, there are dozens of services available today, from Netlify to Firebase, but Microsoft has never really been one of those options (apart from hosting your app using an Azure Blob Storage) until last month, when they released their new service in preview, <strong>Azure Static Web Apps</strong>.</p> <p>Here are the reasons why I think it is worth considering.</p> <h2>Get your app hosted &#x26; running in no time</h2> <p>If you haven’t used Azure before, you’d be surprised how easy it is to get started, and not only will you get set up in no time, but you’ll benefit from 12 months of Free Tier on most used services. Check it out <a href="https://azure.microsoft.com/en-gb/free/">here</a></p> <p>Once you’ve picked your favourite frontend boilerplate and pushed it to Github, the only thing left to do is specify in your Azure Portal which branch you want to use. In less than a minute, your app will be hosted and available publicly.</p> <p>In the background, <strong>Azure Static Web Apps</strong> service will generate &#x26; push a Github Action template to your repository, which will be triggered every time a branch is updated, creating for you multiple environments (even for each PR) without having to worry about CD. You can then enhance this action to add your own unit and integration tests.</p> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <a class="gatsby-resp-image-link" href="/static/20f17d91e16ce856deb8fe8e9e8e3874/50383/github-action.png" style="display: block" target="_blank" rel="noopener"> <span class="gatsby-resp-image-background-image" style="padding-bottom: 41.62162162162162%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAICAYAAAD5nd/tAAAACXBIWXMAABYlAAAWJQFJUiTwAAABn0lEQVQoz3WRWS8DURTH53vgAaUdrbaWbtNWF+XNEkV4kHhE0qSiaPAulgQPeBEP4iNYgm9jCWqJqllq2vk7905aQkzyyzn3TM7vnjMjaEUdilqErGp4y7+j8CHjQ1YhKxr0kkGUsba5hZnULOYWljCbySJNsHw+u4LM4jKP7JxKZyAAZWiFF7w83kItvELOP0MhymoeladvcBgW0Q1Xp4TWdj+PLS4P7G4vHG0+ODsCFL2oqbdCGNq7wcTBHaaO7jF9/IAZYvLwHsn9WwztXnPc8VFY7Q54pBg6/GE02lxoanGjwdqK+mYTlrOasH2Zw8bpHTbPc1g9ecT6mcnWRQ47V08cqXeEGkRIkR4EIwl4AhGSR+ALxfiZRX8ozqenlQ1aSidKJgaLZZh1g6+cHBlHrUXkjYGubhInEIr1ksAHq6O9Cp9Q0XSoxRK0T/0b+lEy1dk7hfL+5BjqmkQEwqbMK0Xpm7r4moxGm7OKUCTBf3zqJc4AE9KEbC1/OA6pK0FyM2dTMRG7gCFUmn7zV2jjkmC0B6KzkzczGYs/hV9iIJ+mcW3FkgAAAABJRU5ErkJggg==&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Github Action" title="Github Action" src="/static/20f17d91e16ce856deb8fe8e9e8e3874/50383/github-action.png" srcset="/static/20f17d91e16ce856deb8fe8e9e8e3874/1d79a/github-action.png 185w, /static/20f17d91e16ce856deb8fe8e9e8e3874/1efb2/github-action.png 370w, /static/20f17d91e16ce856deb8fe8e9e8e3874/50383/github-action.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </a> </span> <h2>Take advantage of the Microsoft Developer Experience</h2> <p>We have to admit that Microsoft have done an amazing job with VSCode. It is today, without any doubt, the most used IDE for web developers (haven’t tested it yet? check it out here <a href="https://code.visualstudio.com/">https://code.visualstudio.com/</a> - In Theodo UK, we have decided to make it our default environment for web projects) And with such tools, the more popular they are, the more their ecosystem will become rich thanks to its community.</p> <p>In our static web application example, it takes 2 clicks to download a <a href="https://marketplace.visualstudio.com/items?itemName=ritwickdey.LiveServer">LiveServer extension</a> and have a server running your code locally, with hot reload. If you’re then planning to retrieve data in your app from an API, check out their <a href="https://marketplace.visualstudio.com/items?itemName=ms-azuretools.vscode-azurefunctions">extension</a> which allows you to simulate your Azure Function running locally, test it, and deploy it all of that from your IDE.</p> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <a class="gatsby-resp-image-link" href="/static/b26901e46811728d626f3859ea38eb7c/50383/vscode-extensions.png" style="display: block" target="_blank" rel="noopener"> <span class="gatsby-resp-image-background-image" style="padding-bottom: 26.486486486486488%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAFCAYAAABFA8wzAAAACXBIWXMAABYlAAAWJQFJUiTwAAAA1ElEQVQY04WQTU/DMAxAwy+A8yRga5MOtqoCrfQrbdN1SeHIbUyD//8zHikVggMbh6dn2ZZtWaRlTalrsqIiL3/IfnmKNUWlyf9BRJkhLresi45V3n35Pp9yo79zd1lLlDZET83kE4ir/JXr7RuBe2dm9tzuDtz0BxbuyKzbM7dHwuGDS98n1gMX8Qsifj7Nw+OGtu0o/blNa2hNR+VfoOsGFS0JAkkYKhajZYRUy7OIdJPirMP4YXZnGdyA1hrnHH3fY4xfMtasJUkSP1yilELJv/kExMiftpBqJEgAAAAASUVORK5CYII=&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="VSCode extensions" title="VSCode extensions" src="/static/b26901e46811728d626f3859ea38eb7c/50383/vscode-extensions.png" srcset="/static/b26901e46811728d626f3859ea38eb7c/1d79a/vscode-extensions.png 185w, /static/b26901e46811728d626f3859ea38eb7c/1efb2/vscode-extensions.png 370w, /static/b26901e46811728d626f3859ea38eb7c/50383/vscode-extensions.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </a> </span> <p>Finally, you can write your app in TypeScript, framework designed by Microsoft, to type your JS code on your frontend and API, for a smooth and reliable experience.</p> <h2>Rely on Serverless to scale your app</h2> <p>If you’re not familiar with Serverless, we could summarise it this way: it is the concept of not provisioning servers, relying only on cloud services, to host your code and scale it up and down as needed automatically, greatly reducing the maintenance cost. (want to learn more? check out <a href="https://serverless-transformation.com">https://serverless-transformation.com</a>)</p> <p>As I mentioned just before, <strong>Azure Static Web Apps</strong> service allows you to use an API to retrieve data in your app. To do so, Microsoft give us the possibility to use Azure Functions, in JavaScript or TypeScript, to then be called directly from your frontend code. Once your environment variables defined, you just have to configure your SSL, your domain name and your app is good to scale!</p> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <a class="gatsby-resp-image-link" href="/static/2d16bdd5791594bf41bef4eec060d419/50383/Azure-Static-Apps.png" style="display: block" target="_blank" rel="noopener"> <span class="gatsby-resp-image-background-image" style="padding-bottom: 59.45945945945946%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAMCAYAAABiDJ37AAAACXBIWXMAABYlAAAWJQFJUiTwAAACj0lEQVQoz2NgYGAQYmZiEhIXE2Pn4+NjBvIZBAQEwHjuvHkgLsO3DU2M/n07BEP7t/JkTtnAOXXOQtYF8+ayzp07j2XevHmsIAxSNw+q3pCVldVDWEjIkp+fXx7ZwJypGxkY1P6DFal2nTaS7L6mLjrhoSBDyjdGBiQANIgF2UAGYVUzRlFpBQY+Hm7GyI2fGBxbtoHFHWt3gOmD564xVmfGcsfHxwunJsULgMTWz53IPGveApALmefNnQt2YdyKCwwo4P///3B2bv8aOHv+wqVM3X0TuLIyM1jzMlJYHFoPiGbP2sV6bnI2w7wFC0GGsoPU3QbpL6hsZrCu3MrGwJkC1py3+zZb8u4HYC9Jhq9kZnBfzfT/+lyGuQuXsMMsMK0/WG/eekICxG6dtpR54eLFbHDbw5Z/Y7ZtPmGUN3Un+4HJJeyO7cfq7VsPaYPkzOsP5JnVHfQDsbunL+YH0SHBQSLeIbFy/sHhYp7OtsyTth1j3t2cxZO76YJeyp6HtQyBoVG8YdmNEuERUQJVubEs1umTjexSewVBmh3yZynY5s0R+/8/n2HarHmcrKxMDP4BQax9XuJssd62rLpWrowLV65j2jCrlz1i4Tkun7mXVBmCfVwZzYGaPVydUGKOoQ0Rnr3z1jHBwgkEzhyN0G+9cQvszZnzFzLPXrCInQEn2AY06BfEMO3VEHrB3FlMc+bMBWty8w5izssLEQuKiecA8ecAY3k+1DLfqScZGYKCgrBiEOjq6oKlM6CBczhWr17DUlhYwBMfly6Qk5XFt2TpMpa5c+eyAeXBrp0/fz4DXgBUDKOZgIqZcKmDJWyYegY8CmE0I1AxO9BQttmzZ7PNmDGDHUSDXAYUY0fOegBbXfIs19e7VwAAAABJRU5ErkJggg==&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Static App Ecosystem" title="Static App Ecosystem" src="/static/2d16bdd5791594bf41bef4eec060d419/50383/Azure-Static-Apps.png" srcset="/static/2d16bdd5791594bf41bef4eec060d419/1d79a/Azure-Static-Apps.png 185w, /static/2d16bdd5791594bf41bef4eec060d419/1efb2/Azure-Static-Apps.png 370w, /static/2d16bdd5791594bf41bef4eec060d419/50383/Azure-Static-Apps.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </a> </span> <h2>Conclusion</h2> <p>Since Microsoft have gone down their open source journey, they are becoming a strong contender for a move to the cloud. Want to get started? Check out their <a href="https://docs.microsoft.com/en-us/azure/static-web-apps/getting-started?WT.mc_id=staticwebapps-docs-cxa&#x26;tabs=react">quickstart</a></p><![CDATA[Don't overestimate the importance of performance when choosing a stack for your Web project]]>/2020/05/dont-overestimate-backend-language-performance-for-web-projects//2020/05/dont-overestimate-backend-language-performance-for-web-projects/Thu, 14 May 2020 00:00:00 GMT<p style="font-size: .8rem; font-style: italic; text-align: center">Understand how a suitable backend language can make a server handle higher loads and help reduce operating costs – or not.</p> <p>A few months ago, a colleague of mine gave a talk detailing how great his recent experience with Go had been. As a side-note, he mentioned that <strong>Web applications written in Go were able to handle way greater loads than those powered by its mainstream Web competitors</strong>. This intrigued me: if this performance gap does exist, what is it due to, and <strong>is it significant enough for me, a Web developer, to actually care about it?</strong></p> <p>After a bit of research, I was confident I had a better understanding of the factors at play in the performance of Web servers. Still, <strong>I was lacking some quantitative insight</strong> that would allow me, at last, to decide <strong>whether taking these differences into account would help me bring more value</strong> to my next project. This is when I chose to run a small home-made <strong>benchmark</strong> comparing Node, PHP, and Go – three technologies we use at Theodo – that I will use to draw the final conclusions of this post.</p> <h2>What is performance anyway?</h2> <p>The first step of my investigation was to get sure of what I meant by <em>performance</em>. Although there is no single definition of this concept, I am confident that the following is relevant to my needs as a Web developer:</p> <p><strong>Performance is the ability of a server to handle a great number of requests during a given period of time.</strong></p> <p>Thus, a server that performs well according to this definition will be able to run on less powerful hardware, reducing operating costs and environmental footprint. </p> <p>It is also correlated to other definitions of performance you may have thought about. For example, the faster a server can respond to a request, the more requests per minute it should be able to handle.</p> <h2>What a performant server is made of</h2> <p>The next thing to realize when trying to understand how a particular technology may influence the performance of a server is that during the whole process of serving an HTTP request, <strong>time dedicated by the CPU to executing code actually written by the Web developer or by the Web framework developers is often extremely small</strong> compared to:</p> <ul> <li>Time spent performing <strong>heavy computations</strong>, such as hashing a password, manipulating images, etc.</li> <li>Time spent <strong>waiting for network calls to terminate</strong>. Our server is most likely communicating with distant APIs and databases, or with the local file system. This takes way more time than executing a few thousand instructions on a CPU.</li> </ul> <p>As counterintuitive as it may seem at first, <strong>heavy computations will not make much of a difference</strong> here. "Slower” interpreted languages will typically call well-optimized binaries under the hood to handle those kinds of tasks.</p> <div class="gatsby-highlight" data-language="js"><pre class="language-js"><code class="language-js"><span class="token punctuation">[</span>Summary<span class="token punctuation">]</span><span class="token operator">:</span> ticks total nonlib name <span class="token number">0</span> <span class="token number">0.0</span><span class="token operator">%</span> <span class="token number">0.0</span><span class="token operator">%</span> JavaScript <span class="token number">72</span> <span class="token number">63.7</span><span class="token operator">%</span> <span class="token number">100.0</span><span class="token operator">%</span> <span class="token constant">C</span><span class="token operator">++</span> <span class="token number">1</span> <span class="token number">0.9</span><span class="token operator">%</span> <span class="token number">1.4</span><span class="token operator">%</span> <span class="token constant">GC</span> <span class="token number">41</span> <span class="token number">36.3</span><span class="token operator">%</span> Shared libraries</code></pre></div> <aside style="font-size: 16px; font-style: italic; text-align: center; margin-bottom: 1.45rem;">Profiling of a Nodejs app that hashes a password using bcrypt. Virtually no Javascript is executed (a tick is a unit of time).</aside> <p>What will make a difference on the other hand is the way <strong>the server uses the waiting times</strong> during the processing of a request <strong>to start handling other requests concurrently</strong>.</p> <p>Scheduling how the CPU will handle numerous concurrent connections to the server is a complicated problem that can be addressed in a number of ways:</p> <ul> <li>Typical PHP servers will let <strong>OS threads</strong> handle the task. To put it simply, <em>threads</em> are the mechanism that operating systems use to give us the feeling that they are processing several tasks in parallel. Thus, threads are also natural candidates for managing the concurrent processing of HTTP requests – <strong>each thread being responsible for a single connection</strong> – even though this is not what they were primarily designed for. This unfortunately comes with some <strong>performance issues</strong>: each new thread consumes quite a <strong>lot of memory</strong>, and time spent switching from the execution of a thread to another is also non-neglectable.</li> <li>Technologies such as Node or Go will keep the processing of all the requests in <strong>a single or few threads</strong> but will use their <strong>own user-land mechanisms</strong>, such as an event-loop (Node) or Goroutines (Go), to address the scheduling issue. Although these two mechanisms will make Web developers write very different-looking code, they have in common that they do <strong>not suffer from the same overhead as OS threads</strong> do when opening a new connection or when switching from a connection to another.</li> </ul> <h2>Benchmark</h2> <p>We now know that <strong>thread-based Web servers should in theory be outshined</strong> by their counterparts that use a more suitable concurrency model. <strong>But is the performance gap wide enough to be of any importance to developers?</strong> Let’s find out!</p> <p>To carry my little experiment, I set up a server and wrote three different routes in PHP, Nodejs, and Go (more details about versions and my exact protocol in <a href="#appendix">appendix</a>):</p> <ul> <li>When <strong style="color: #195e83">route 1</strong> is fetched, the server <strong>hashes a string</strong> then returns OK. </li> <li>When <strong style="color: #24c49c">route 2</strong> is fetched, the server performs a call to a (very) distant server, <strong>waits</strong> for the response, then returns OK.</li> <li><strong style="color: #ef6f6c">Route 3</strong> is a <strong>mix between route 1 and route 2</strong>: when it is fetched, there is a 1% chance that the server hashes a string, else it performs a call to the distant server. It emulates a server that performs a lot of I/O (database queries for instance), and some CPU-intensive tasks (such as hashing passwords to identify users).</li> </ul> <p>To compare performance, we’ll put some load on every route, one by one, for every language, and use the following comparison criterion:</p> <p><strong>Reference load = the maximum number of HTTP requests a server can handle in a minute, such as 95% of requests take less than 3 seconds.</strong></p> <p><strong style="color: #195e83">Route 1</strong> falls into the <strong>heavy computations</strong> category, so we expect to see little difference between all three technologies:</p> <div class="image-container"> <div style="width: 500px; margin-bottom: 1rem"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <span class="gatsby-resp-image-background-image" style="padding-bottom: 62.16216216216216%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAMCAYAAABiDJ37AAAACXBIWXMAAAsSAAALEgHS3X78AAAB6klEQVQoz4WSXU8TQRSG+7v0Sv0B3qHxQq6aEG1N/IArv4ixiilVjBHvDReoVwajMQaMERTrVyi11KQUttV+JahQ2p3dmT7OTLukrVU3eXdnZue885xzJsSAp9VqfyvVbRqNBrXdTer1HaquS8MV/OsJuXqTEAKl1J6kVPi+1CZNhOvhSx8UCAk72nBX73el7IkxammSkHn1SzP2nFqu5yj9SLP6M8eFrEPkQ4aZ0pb9p/piQ77vayJpJ0q1bL6GbuntOnNPUjx9+Y6bS0NcebGfWPIkIxmHY6k8d5yKNZRqgKFRkKpZbDYFVyeeEx6ZZfTSDPHXQ0wuH+RaMkJ0zWF4dYPpQtUeLv+WcnczhC5W4vYCZ8fmuHj9IZPaMP7mgDWMaMPjgeEfxelrSjt1ZQnjt+Y5fe4x52OzlrDf8K5jCHXzdIzsyBL2XpdWh9DvMnww0DAgtE3pJvQ8D6N2U5RNotn0/ks4bQgZUMNg0k3oasLE1AJnRts1TCweJbF8iIn3UU5lCwynN7lXrPVcmz1Ds2i6bCiF8OzYXOjYjWeciD5i7PJ9YvOHib3ax/himPBKjiMfvzK1/s0EIjpxpdqWHdsuB2l3K79RJq1JvqzlyRY+kS0myRRXSJUqfP5epvBrG9kBCWQofwMfsGAZ8ocmuQAAAABJRU5ErkJggg==&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Time to complete one isolated request (ms). PHP: 320ms. Node: 350ms. Go: 380ms." title="Time to complete one isolated request (ms). PHP: 320ms. Node: 350ms. Go: 380ms." src="/static/6cb64dd9ee9594544ceb355067e77422/50383/1-isolated.png" srcset="/static/6cb64dd9ee9594544ceb355067e77422/1d79a/1-isolated.png 185w, /static/6cb64dd9ee9594544ceb355067e77422/1efb2/1-isolated.png 370w, /static/6cb64dd9ee9594544ceb355067e77422/50383/1-isolated.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </span> </div> <div style="width: 500px; margin-bottom: 1rem"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <span class="gatsby-resp-image-background-image" style="padding-bottom: 62.16216216216216%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAMCAYAAABiDJ37AAAACXBIWXMAAAsSAAALEgHS3X78AAAB70lEQVQoz41S30sUURjdfyvoJfoDhBD0qXrQklqDMoXAwjTaBnYVJQp8DoPVfLAXwaCQRdcfsEjuutHO7s6dlXLBRNHZmXvv7Ol+d5x1XJfogzOXgfude75zvpjneSD4vn8FjuNBcAGXn0EKCUf4aHAOIaVG+/1ms4kYfaKg4qo5/SmH5NQyPnzcxKnjgMuGanIRrfZeTSjVSwT6oVeoPE/ASH3BQHwBz16mMbsxiLfrvZgrTGDj6ASrB4f4cdYgRvjthEIIcDUGkUkZyHZdjqmZrxga/ozniTSM1Vsw1q7j1eZ93Nsto3trDzPsQBPKTiNfjBCcnEtMKsLHQ0sYfZ1GKtOLVPYm3mzF8ajIcLfA8N6uBz24XLFoKMHovlY4Oa0InwaEyUwPktkbivAh4kULt3ereHeukMKRrV6lsN3kSwr/RXiuUHsYVUjM5ONFKOSh+A+FdX1XRvzTCqOGhoSUMoXyRIXyIjHf8tDYDjy8k7daHmpCCiccOSSKEpOHRnIFDwYXMTI2h8S3LiQy1zCe7UN/voKenTKmq79bCmk76n+O1f6KwENamzAcguu6sGuHKBb3sfP9J/bYNgpsHXk7h9z+LxSOjlE7Vcse6QmDjZVKJXSCZVXAWEWfVtlG1awpMDCzBLtswjLNjn1/ATX1WfBWwCIeAAAAAElFTkSuQmCC&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Reference load (requests/min). PHP: 188. Node: 170. Go: 158." title="Reference load (requests/min). PHP: 188. Node: 170. Go: 158." src="/static/51e5cc5ec78b499d22bb305f5dcb07c5/50383/1-reference.png" srcset="/static/51e5cc5ec78b499d22bb305f5dcb07c5/1d79a/1-reference.png 185w, /static/51e5cc5ec78b499d22bb305f5dcb07c5/1efb2/1-reference.png 370w, /static/51e5cc5ec78b499d22bb305f5dcb07c5/50383/1-reference.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </span> </div> <div style="width: 500px; margin-bottom: 1rem"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <span class="gatsby-resp-image-background-image" style="padding-bottom: 62.16216216216216%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAMCAYAAABiDJ37AAAACXBIWXMAAAsSAAALEgHS3X78AAAB7UlEQVQoz62SW2sTQRzF92v1UfwAgg9eXlTQ4i0qXuqTWlRIGhrjpaBfQPqgPtkiCKVara0hhdTWNgnF6CZEkyZewF27t9mfM7PZ3LBvDpwdZnb+Z845/zEcx8F1XYQQA/D9gN/WDo7jsuNaeJ7HH9/HlRg+GyMMQwz1GQYoDA4RCrkt9B/xj5oYhi9vDIJAL+KDQSDI5U1mX6yytFyBoI85DLvzroTKjpKsiNSm43hMTM5x/OQ012/PUKzn2dp+T/XXJrbnY8nzjhSxq+Xhyz0v4O7Uay5fneVG6inphf3cmh8hvXKa8XKVC4Uy040fUc1QNIZqSNyUyLrQCrP3X3HxynOuJZ9wZ/EQ2dxeUvmznNo0Obz+mSmzqTP1ZY1yFsdmMBBP2FWYfdAjzCweJLO8hwlJeK5U5ejHLzyqbXeaFQ6oNFR+Cropopdhv8LM24hQKUwUTY5IwofVpjYcdPPrZNgfaKzQdX2d4aWxGZ1h9l1kOb2S4LzM8NiGKRW2egr1y+izrBZRt9UT8rFth9TkS84knjE2/pjk/D6Sb0a4uXSC0fVPHChsca9Sl4WyplPXaP/ElU51l5Vl1Rg127ZFq9WiJJWsyfA3ihXKtQLFWo5SfY0PXxusfmtSbjT53m5jWRbx01PCDP7z+AvbV283koTpZwAAAABJRU5ErkJggg==&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Total RAM consumption on reference load (Mo) - 980Mo available. PHP: 320Mo. Node: 270Mo. Go: 255Mo." title="Total RAM consumption on reference load (Mo) - 980Mo available. PHP: 320Mo. Node: 270Mo. Go: 255Mo." src="/static/18f409a18a072da3e1049b1a97eaa41c/50383/1-RAM.png" srcset="/static/18f409a18a072da3e1049b1a97eaa41c/1d79a/1-RAM.png 185w, /static/18f409a18a072da3e1049b1a97eaa41c/1efb2/1-RAM.png 370w, /static/18f409a18a072da3e1049b1a97eaa41c/50383/1-RAM.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </span> </div> </div> <p>The <strong>small variations</strong> we observe can probably be explained by slight differences in bcrypt implementations. An interesting fact to notice here is that the reference load could have been predicted by dividing 60 seconds (1 minute) by the time it takes to complete a single isolated request (PHP case: 60 / 0.320 = 188). This means that our server is simply <strong>processing the requests one after the other</strong>, and has <strong>no opportunity to parallelize anything</strong> since there is no waiting time while handling a request.</p> <p><span style="font-style: italic">So far, our results match the theory. Great!</span></p> <p><strong style="color: #24c49c">Route 2</strong> is much more interesting since there is now <strong>plenty of time for the server to parallelize requests</strong> while it is waiting for an API call to return. Let’s see what happens in this case:</p> <div class="image-container"> <div style="width: 500px; margin-bottom: 1rem"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <span class="gatsby-resp-image-background-image" style="padding-bottom: 62.16216216216216%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAMCAYAAABiDJ37AAAACXBIWXMAAAsSAAALEgHS3X78AAAByklEQVQoz52Ry2tTURCH7x/lzp24daGrulIolSLEhY8uixWsBEyR4kLcFxW78NGVz/hIIaCBPtSkatD0ttrexDxu7+Oce5LPc45JG6MN6IEvwyUzv5n5jRPHMYZOp0O73f6NMBRIKYnkjo1hkiA1g3k9jIZjfvZjvzesxkl0R8OwpH/BCpp1BldOErWH0k2V7Eb9n1KoYSv/7/tbpSOEsEcxHZTubKKUCbnFz9y7X+BxdolX5ds8Ld7khbvAg02PW2WXfMOHbo3qRjvhoNnmSalIX33C2PgdLkzOcTl7hEvPDzCVH2V0tcyxtyVmK9s2t22866t3jH9mSvOh1C8fhEi4dj3LuYkFJq/cZeb1cWbyh0m/SXGmWOHku6/cWN+2S6s+/0x0eoYOTmgEz55/qAXnyeRGyOQPWcFUaZ0TRtD19ibsO6hduXdpIfQlE0kQRqQzjzidmmfi4hzTz44y/fIgU7kxTi1/YqSwxmx5wxQidF0cCza9H8R6013BQSrao/cfKhRLX1hzl/i4UaDorrD6fYvlb1u4LZ+2Urv59jgap9Fo0Gw2/yAIfKIoIAoDQj/WCMKdmMhvWYJWy+bV63Wq1Sq1Wg3P8/gJSneCXquD7ZQAAAAASUVORK5CYII=&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Time to complete one isolated request (ms). PHP: 390ms. Node: 390ms. Go: 390ms." title="Time to complete one isolated request (ms). PHP: 390ms. Node: 390ms. Go: 390ms." src="/static/4984862455bd11fd8c24b36e478f36ca/50383/2-isolated.png" srcset="/static/4984862455bd11fd8c24b36e478f36ca/1d79a/2-isolated.png 185w, /static/4984862455bd11fd8c24b36e478f36ca/1efb2/2-isolated.png 370w, /static/4984862455bd11fd8c24b36e478f36ca/50383/2-isolated.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </span> </div> <div style="width: 500px; margin-bottom: 1rem"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <span class="gatsby-resp-image-background-image" style="padding-bottom: 62.16216216216216%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAMCAYAAABiDJ37AAAACXBIWXMAAAsSAAALEgHS3X78AAAB0klEQVQoz42Tz2vUQBzF8z9586w38apCFbGXuqBVe+jBFhRst1Cq4F1Q8GBPFVetB4tIrbJIu26zjUX3h213u/VHKTZNNskk+TiT3chsTcEHjyHM5H3f9813DM/zsG2bOI6JoqiPjuMRBAGuv5+sjhCEYUiCjPNKw9A/DjMLthTdPXA4kGvWP4aqqKpnbaaM4ihx5MnCt782GVgqMV3fRqiiyb4m6Ps+ruv+Y1+nCAWRpCN8blZbDKxtMFFvE0gzsdwP9ZY5At1M4+SwjvFqm/Nmg3xjh7AXix6OkV6KqqDaT1cd678WWbAe8e7bHKNfqlyobDIpW/a184qZDpUrhdVKmwcP3zD7pMz9tznGXh9jYvEUw9YnLq61EkGROtQsGslYyAzTsRGi6+5pwWRw6DFXhwtMvchxb+Ukd4tnuWaVpWBTZrhN0MtNn5K+W9YFX76yuDE6x9j4AtPzOWY+nmCmeIbr1iqXrBb5Q4J9LesjEobdS3j23GToyiwjI/PcKQySLx5n6v1pLldKnDO3uFVrIXrxqJh+7u3LaQi7gkIOaafT+Uvf99j5vsfn9SalUg2ztkxlYwmz8YHlrU3KP3ap/7ZxtX8UlY6R9SKOeCT/hT+t/4DusOGJjgAAAABJRU5ErkJggg==&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Reference load (requests/min). PHP: 32000. Node: 39000. Go: 69000." title="Reference load (requests/min). PHP: 32000. Node: 39000. Go: 69000." src="/static/86c401dce0bd78e87153c95f0bcbc079/50383/2-reference.png" srcset="/static/86c401dce0bd78e87153c95f0bcbc079/1d79a/2-reference.png 185w, /static/86c401dce0bd78e87153c95f0bcbc079/1efb2/2-reference.png 370w, /static/86c401dce0bd78e87153c95f0bcbc079/50383/2-reference.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </span> </div> <div style="width: 500px; margin-bottom: 1rem"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <span class="gatsby-resp-image-background-image" style="padding-bottom: 62.16216216216216%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAMCAYAAABiDJ37AAAACXBIWXMAAAsSAAALEgHS3X78AAABtElEQVQoz31TTUvDQBDNf/KHeNKD4ElQKqiIIqLiF3r1rBehWNSLloLFuz2IX9So1IJVa01aa6za0iZNNslzZ9NoGrUDw8LuzJv3ZmYldLByuYp6vYFKQ4XeaODNtFBvNsWb+0+OZJomdF2H67pwHEe4bdtekuulNVkdlmVBZwwmdz8u7BQv+UB0+k6Wf3rH2cUD5GsFTcMrgNZbMDbsEoExXtW/sG1HJEVjpxgY2sLkdByq+inu7FDhPwFJHsnxmTLmsYltn2FsIo7ZhSQH/PAAeex/ctskB81x3BbgOUbG9zAzf4BiqdrW004mkVzDMFpyiS0TD1ucoQ+otBhSLKMY7izElnIFw3AF6lOYYalUQ8ddCTOk1Qn38IdhEgVFI36omgZiqoa1zD0Srx+/+vc9ZZ9ucMqxnXOMTuxjbvEQG0czWD/ux0Z6CpFMFr3yI1YeinAC022T7IMGe7gZPcbg8C5fmwSWkj1YTnVhKdWNyPUV+q7yWM0psLg6kcPPcuVTnGLKtDYkm1x8tcob7nLPuJTvcXPziGxBxm3hBFklDVlVkS6WkSm9QNM01Gq173wC/wKJxoP8mWtToAAAAABJRU5ErkJggg==&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Total RAM consumption on reference load (Mo) - 980Mo available. PHP: 890Mo. Node: 220Mo. Go: 300Mo." title="Total RAM consumption on reference load (Mo) - 980Mo available. PHP: 890Mo. Node: 220Mo. Go: 300Mo." src="/static/46d8b960c12d23be1cdfe931265c749a/50383/2-RAM.png" srcset="/static/46d8b960c12d23be1cdfe931265c749a/1d79a/2-RAM.png 185w, /static/46d8b960c12d23be1cdfe931265c749a/1efb2/2-RAM.png 370w, /static/46d8b960c12d23be1cdfe931265c749a/50383/2-RAM.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </span> </div> </div> <p>We notice that the time it takes to handle <strong>a single isolated request is exactly the same no matter the language</strong>, which seems natural since the bottleneck of our route is a very long network call. However, huge differences arise when we start putting some load on the server:</p> <ul> <li>Our Node server can handle about 20% more load than its PHP counterpart, which is significant, but definitely not as spectacular as the <strong>1-to-4 ratio in RAM consumption</strong>. This seems to validate that <strong>thread-based servers are more memory-intensive</strong> than servers that use a more specialized concurrency model!</li> <li>Meanwhile, our <strong>Go</strong> server is able to keep a low memory footprint while handling almost <strong>twice as many requests per minute</strong> than the former two. I’ll admit that I have yet to find a satisfactory explanation for this result, and if you do have one, I’d be very grateful if you could take the time to share it with me!</li> </ul> <p>However, a real-world server could very well need to perform a few CPU-intensive operations among all these network calls, which is what I tried to emulate with <strong style="color: #ef6f6c">route 3</strong>:</p> <div class="image-container"> <div style="width: 500px; margin-bottom: 1rem"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <span class="gatsby-resp-image-background-image" style="padding-bottom: 62.16216216216216%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAMCAYAAABiDJ37AAAACXBIWXMAAAsSAAALEgHS3X78AAAB9UlEQVQoz41SS0sbYRSd31V34tqdFcGFihWN1CeK2o0gjrZVcV/BTcFN6wNMIlboorRqfRC1k4fYhKBJlNSAOu9vJsfvzqPJBBE/uMzjfufcc+69Ap451zd3kGUFt8oVVEVBUTfxoOlOrlwuP4kRDMPgIBm2bf8Py7KdJL3T0ZkMuqcyBouHlwxgKKiI4BJYzgeF7VUu/pORSOaRTBWga5ZLwlMqL3bPVaqsgqkOoVaybbuE4WgcXb2fEepbwUW66PwjhTPpPDp3T7CYKcArg2rzgmmaUHh/fNmMudc2I38wMLKGsYltfPo+jaX9N1g+GsVgMo5W6QpTf/MwyBk5rLXMeGVfst+/cFRC3/AXjI9/gxjuhLj7CrO/GjEQj6FdykHkShm5eblln3AbH6IhLBw2YGG/mROeoI0rJELL63fAMg1E13V3INWWHcKvDuH7SA/mDuoxv9eEfo9wmhOans3AlIlQ07QnelhR6BPO7b0OEno9DBDWWvb3NbKV4ENZxcS7HXzcCmGeLP9uxmDiFO3xHGb4lO0AzgU6hKSSFpcmTvbprG3E0P12BUNDm5ha74D4sw7ij0b0xA7QEstgMpWFyjGMMPxZKJagm0aFkMj8sCyG29ID0pkbSFIWqewZznOHSF4e4yzPl710h5yiBTC0KWT7EQ+rZvgH1vzNAAAAAElFTkSuQmCC&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Reference load (requests/min). PHP: 13200. Node: 11000. Go: 12800." title="Reference load (requests/min). PHP: 13200. Node: 11000. Go: 12800." src="/static/8941a1747157ce7346a9e39d057deb51/50383/3-reference.png" srcset="/static/8941a1747157ce7346a9e39d057deb51/1d79a/3-reference.png 185w, /static/8941a1747157ce7346a9e39d057deb51/1efb2/3-reference.png 370w, /static/8941a1747157ce7346a9e39d057deb51/50383/3-reference.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </span> </div> <div style="width: 500px; margin-bottom: 1rem"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; "> <span class="gatsby-resp-image-background-image" style="padding-bottom: 62.16216216216216%; position: relative; bottom: 0; left: 0; background-image: url(&apos;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAMCAYAAABiDJ37AAAACXBIWXMAAAsSAAALEgHS3X78AAAB6klEQVQoz5WTTWsTURSG87fcuPEPFEREVxasEpEq1S6EWhCZSoJTq6Yu3QhujIqKIH5AP6Raq1SjtRO1Nh9tmjFJNZqZzvfjvRkntul00QMvhzuc897n3nMnQUwEQZirepNWy6BhrmAaBjXL4Y9lhTXER8K2bSxR5Pv+FskIs8+G08JxHEzXxRbqro0UCJLE5kUk3w8oldfRtArLhQa+132CYEe1CaWkqed54qMv1i5Xrk3Ql7zFxZEnNH+b4og+rrcz3TbCaGcZrutxNTPJqbP3uZR+jmG4RBfniRopf7eEY8Kwf+AeSuop+rouhtHAtH9tmZy3G8KxzASnzzzg/MhdbswcZ3S6h5u5czyu1sgWK8w1jdj7TMjpxRJel4YPGVLukJ46QPrVXpQ3xzj6aYn9776gFqrCLezxOr0xU+4mHFKyXJ4+hPp6H6m5JP1agd7PJcZLevi0On3/CF3xriRl+FxCY8cRQxmfZGDwEcPiyOrLw4zOCsO3J4RhkSMLRTLlH22TECj4Tyh3kYvIWGZzwyKlPiN5Msvg8G2UFz0oU3u4MNNLX+4rB+e/oS6tEsge12n3Ves/sUVOhEROR4b8xWp18vkSuY/LLCx+J78yz2J5Fm01x4fKGu/XdDShRr0ufs1WB0aC/QW3WHOkwtVOCQAAAABJRU5ErkJggg==&apos;); background-size: cover; display: block;"></span> <img class="gatsby-resp-image-image" alt="Total RAM consumption on reference load (Mo) - 980Mo available. PHP: 440Mo. Node: 270Mo. Go: 245Mo." title="Total RAM consumption on reference load (Mo) - 980Mo available. PHP: 440Mo. Node: 270Mo. Go: 245Mo." src="/static/0462e15433d132824b306335b3cca2d8/50383/3-RAM-2.png" srcset="/static/0462e15433d132824b306335b3cca2d8/1d79a/3-RAM-2.png 185w, /static/0462e15433d132824b306335b3cca2d8/1efb2/3-RAM-2.png 370w, /static/0462e15433d132824b306335b3cca2d8/50383/3-RAM-2.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy"> </span> </div> </div> <p>The most noticeable thing, in this case, is PHP’s memory consumption, and it is yet far from reaching any critical level. Reference loads on the other hand seem once again <strong>fairly similar</strong>.</p> <h2>Let's talk about money</h2> <p>All that is very interesting, but what matters in the end is how much money you will have to spend to run your application.</p> <p>As an example, let's consider a large Symfony (PHP) project we are currently developing at Theodo. It handles about <strong>50 million requests</strong> a day and runs on <strong>32 machines with 8 cores and 32Gbytes of memory</strong> each. A similar infrastructure on AWS roughly costs <strong>$100,000 a year</strong>.</p> <p>Our benchmark says that a more performant language could potentially divide infrastructure requirements by two. This would mean a <strong>$50,000 economy per year</strong>.</p> <h2>Conclusion</h2> <p>These results are telling me that choosing a suitable technology could have a measurable impact on operating costs if my service is <strong>mainly performing I/O</strong>, with close to zero CPU-intensive tasks. Otherwise, my application’s bottleneck will lie in the CPU-intensive code, and the technology I choose for writing my controllers will be of no importance whatsoever.</p> <p>Let's also keep in mind that there are many <strong>more practical considerations</strong> that should be taken into account first when choosing a production-grade technology, as they may have a more significant impact on a project success. For example, a rich and mature ecosystem should come with a greater development speed, and should make it easier to find experienced developers to hire.</p> <p>Note that Web performance in general should definitely be a concern for whoever wants to build a successful product! For that, you can check our <a href="https://www.theodo.fr/expertise-performance">performance experts at Theodo [FR]</a>.</p> <aside class="picture-caption" style="font-style: italic; font-size: 16px; margin-bottom: 1.45rem">Cover photo by Cory Schadt on Unsplash</aside> <details id="appendix" style="font-size: .8rem; margin-bottom: 1rem"> <summary style="display: list-item"><strong style="text-decoration: underline; cursor: pointer">Appendix: more details about the benchmark</strong></summary> The benchmark was run on an AWS EC2 t2 micro, Ubuntu 18.04. <ul> <li>Nginx 1.14.0, PHP-FPM 7.2</li> <li>Node 12.14.0, managed by pm2 4.2.1 (and Nginx as a reverse proxy)</li> <li>Go 1.13.4 (and Nginx as a reverse proxy)</li> <li>I used <a href="https://loader.io" target="_blank">loader.io</a> as a load generator.</li> <li>Code available <a href="https://github.com/mpierret/benchmark" target="_blank">here</a>.</li> </ul> </details> <style> .image-container { display: flex; flex-direction: column; align-items: center; max-width: 100%; overflow: auto; scrollbar-width: none; } .image-container::-webkit-scrollbar { width: 0; height: 0; } </style><![CDATA[Measure the server-side impact of your application with PowerAPI]]>/2020/05/greenit-measure-server-energy-consumption-powerapi//2020/05/greenit-measure-server-energy-consumption-powerapi/Thu, 14 May 2020 00:00:00 GMT<p>According to a report made by <a href="https://theshiftproject.org/en/article/unsustainable-use-online-video/">The Shift Project</a>, the carbon footprint of the digital is estimated to exceed air travel with <strong>more than 4% of the total greenhouse gas emissions</strong>. This number is expected to double in the next five years! </p> <p>Green IT principles aims to reduce the impact of the digital. As a developer, I wondered how these principles could be used for the web applications I worked on, and what could be improved.</p> <p>I first looked at the definition of digital. It includes: </p> <ul> <li>Networks (for instance wifi and 4G)</li> <li>Data centers (used to host the servers and the data)</li> <li>Terminals (smartphones, computers, TVs)</li> <li>IoT (connected objects like a home assistant or a smart thermostat)</li> </ul> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 391px; " > <a class="gatsby-resp-image-link" href="/static/3eb41a76e3893b02a41a2d6f36690403/14e0c/shift-project-chart.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 94.05405405405405%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAATCAYAAACQjC21AAAACXBIWXMAAAsSAAALEgHS3X78AAAD9UlEQVQ4y11U2YtbVRi/L/W9/4GKiC/Fl9oHF1ALUrCiCD64geDaWgRrq6gEBKetLe3UCjqitdpxYMYZLcjoVLFxZrJP9tzs+55JbnKTe2+WkTrTn9930sS0D1/OyT33/O53fsuRuo0mWuUqWqWKKLVSQ7feQFdRYejXYdQVqMUi2rUGoGvYaragbSroVDeh0j6uJu3j9xhL6istpEJhBJwuuC02JGk+aBvQyyWoix/j31N7MH/8Kdx56DKOzqygkC1jp9NBSuY9G/DaHPBQKcUyeoQlMSpX3B8QpSttGMUc2hdfhvrJfbhxRMLFV/dCeuYL3PvmDB46PgtPIIWcLMPvdCPkotrwoJhMDwG51VF1N+swOltof/MStEMSVNM92Dm6Cz+89aAAvOPZM5CemMIB0zz6DQU600EUMWXjI48mPe602YaeT6Lz2T5ox3ZDfXsXdqjD717bB+ngeex+/jz2vHMJe4/9CKs3iRuaBqVQQtjjI07rAkMatNrIRqLw2R3oNDQY4XXo5x6GcWE/OtOPYvvTu7H0wUHcdWQOD7x3GftNC3j65BVcWQ2iGJbhIx4ZkDXIxZKQdDpm0EXkWm0oEuFG1A599hUY378I/fPH0P/qAGq/n8X7sxa8cG4Zz53+FY98NI8lcxDbaouOWxOKazWqzQYkbjMTjsLvcKFeqKBbLUObewPGzJPQLzyO3tK7uP7nSSxeXcXhb9dgmrNgatGBeDwPrVIV/LF1eNRq9SGgUiyhnMpAq5IH231o12agn7ofvYXDGKycwPbyh3D/9QvO/hbC61/+gUtX/dDKZWGzjXUrXOsWONcsyEbjQ1FYbuZyZCFWTzN/jcGyCQPzNHU4hfrKNH52JPDTqgyl0hAGj/kCiFLJxGHE60ctmx/68BbrUPVJ7Z7aRS8TQjd8DT3XPDTHAtKpnPh4NhxGWo6II26pw0b4+dg2o+KHrVJZiJSmJEQDYcFrr9mB0dTIFgoyBOQSx7QKdTkhk02NkzJ6yGMhFqc42SmGsuiie7NzNnGe1grxJEqUjBzNR5G7xdi3d8kWyMcSQrlJbkdcj+ofVSN61DGY6JB/EpThDVKMgx6kbGbI6ImgTGZ1Q3b7EPMHESQD85rHSp3TGh834vEKu6WIBoM/Ojpyh+zCisluD7wOD+zuKP62hWC2UxK8EQKxwUW24NvF53Ci4TUj43HCsWalPV7oZOhhl8r/WTaEuqrghm3ApLMVAi6vAJLpRonS/2wkhhDFjC3C7zPYpAZSd4K7aiaLRCBEJk+jSMSz0mzWHBXfk7zGQmUINBEIDq+smxyORZlUlzPJQrRp5Hm7MsppfTinUa1Uxzc1r93u4/8AbbEcE0TDhikAAAAASUVORK5CYII='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="Shift Project Chart" title="Shift Project Chart" src="/static/3eb41a76e3893b02a41a2d6f36690403/14e0c/shift-project-chart.png" srcset="/static/3eb41a76e3893b02a41a2d6f36690403/1d79a/shift-project-chart.png 185w, /static/3eb41a76e3893b02a41a2d6f36690403/1efb2/shift-project-chart.png 370w, /static/3eb41a76e3893b02a41a2d6f36690403/14e0c/shift-project-chart.png 391w" sizes="(max-width: 391px) 100vw, 391px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <p>In this chart, the Shift Project shows that the energy consumption can be divided in two factors: </p> <ul> <li>45% of the total emissions is dedicated to the production of digital equipment. To reduce the carbon footprint, it is important to keep a computer or a smartphone as long as possible. However, the <a href="https://www.theguardian.com/environment/2013/jan/13/lifespan-laptop-pc-planned-obsolescence">average lifespan of a computer is estimated around 2 years</a>. As developers, we can have an influence on the lifespan of our users' devices by making sure our applications are not too slow on old devices. Indeed, when you need 30 seconds to open most websites, your first thought usually is: "I really need to change my phone"!</li> <li>55% of the emissions come from the usage of the equipment. We can have an impact on this energy consumption by designing <strong>sober</strong> and <strong>low-consuming</strong> applications.</li> </ul> <p>So how can we measure the impact of a web application on our user’s device? </p> <ul> <li>Some websites like <a href="https://www.websitecarbon.com/">Website Carbon</a> ou <a href="http://www.ecoindex.fr/">EcoIndex</a> (for french users) can estimate the <strong>client-side</strong> carbon footprint of a web application.</li> <li>There is also the 1-byte-model which estimates the global carbon footprint. It takes into account the energy consumption of the data center, the network and the user device. A <a href="https://theshiftproject.org/en/carbonalyser-browser-extension/">browser extension</a> exists to see the impact of a website.</li> </ul> <p>The data centers represents 19% of the digital energy consumption and some improvements in efficiency and performance can be found. However, there is no easy way to measure the carbon footprint of the server-side of an application. Therefore, it is complex to know where to begin to reduce the impact!</p> <h2>Example of a web application</h2> <p>At Theodo, we use an internal application to display the developers' current missions and availabilities. It is used by the sale representatives to see which developers are available and to staff them on new projects. One person has access to a CSV file online, and updates it several times a day. It is important to always have up-to-date data on the application in order to correctly staff the developers.</p> <p>To update the application database, a script runs every 15 minutes: it imports the CSV file and parses it, then modifies the availability of the developers in the database. The file, which is quite large, is therefore <strong>imported 96 times a day</strong>. I had the intuition that it was consuming a lot of energy to run this script as many times, and I looked for a way to confirm or deny it.</p> <p>I searched “How can I measure my server energy consumption” online and found out that simplest way was to plug a wattmeter into my computer, run the script locally and monitor the energy consumption. However, like most people, I don’t have a wattmeter at home, and I was not very keen on buying one for this unique usage.</p> <p>I finally discovered a very useful tool: PowerAPI.</p> <h2>PowerAPI</h2> <p>PowerAPI is an open-source project developed by the Spirals research group (University of Lille 1 and Inria) in order to estimate the power consumption of a software, <strong>without any external device like a wattmeter</strong>. The calculation is made based on the analysis of the consumption of various hardware components (for instance CPU, memory, disk…). The objective is simple: design less energy-consuming software by easily monitoring their power consumption.</p> <p>The power meter created by PowerAPI is composed of two components: the sensor and the formula</p> <ul> <li>the <strong>sensor</strong> collects raw data from the hardware while the software is executed and stores it in a database.</li> <li>the <strong>formula</strong> accesses this database, processes the raw data into a power consumption in Watt. The formula can store the results in the database or export it in a CSV.</li> </ul> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/699501db8820cf0b4e5bc4cf856035a4/50383/power-api-diagram.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 37.83783783783784%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAICAYAAAD5nd/tAAAACXBIWXMAAAsSAAALEgHS3X78AAABIUlEQVQoz4VRXU6DQBDmJl7IxEeP4XNP4YNHMF5BExP7YHyo1kgbG22TWlAxWMoCUiqwy+wnu2BKa7CTTPbbnW++nR8DO4yk3MGo4rLmGY+OhDlbYWClGJV4mRI455pouyGOTi7wZHv6LoSAGxEGrwUepkuYZc7YlShoLWr0XiSuTA/dIcPtTCJICAvfRxzHGFke9jtnuHt2EIUhWBDAWhBuJhznvfcyJ4T5BvBCgogqwd+Sm+U3LV5l/zT6N8dQras5KVdY6nPtXBSbQvWsvjMOUZDGUZKiP3aQl1xjW6ApqizNBTIukNeuMBeEJOWlqNCc+8kH9g6PcdmfqpZbdtfY7vanvs/AGMPnfK7jjveFg84prod2u2BTqO19M16N5gegTWnHXLjGdAAAAABJRU5ErkJggg=='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="PowerAPI Architecture" title="PowerAPI Architecture" src="/static/699501db8820cf0b4e5bc4cf856035a4/50383/power-api-diagram.png" srcset="/static/699501db8820cf0b4e5bc4cf856035a4/1d79a/power-api-diagram.png 185w, /static/699501db8820cf0b4e5bc4cf856035a4/1efb2/power-api-diagram.png 370w, /static/699501db8820cf0b4e5bc4cf856035a4/50383/power-api-diagram.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <p>As the sensor needs information from the Linux kernel, <strong>PowerAPI can only be used on Linux machines and not on a virtual machine</strong>.</p> <h3>Install PowerAPI</h3> <p>To install PowerAPI, you need:</p> <ul> <li>Docker installed on your machine</li> <li>a MongoDB database that can be accessed by your machine.</li> </ul> <p>In my case, I decided to execute the project locally and deploy the MongoDB instance on my computer.</p> <p>In the MongoDB database, I created a database (I decided to call it <code>power-data</code>) and two collections:</p> <ul> <li>a collection to store the raw data from the sensor (in my example <code>sensor-data</code>)</li> <li>a collection to store the computed results from the formula (<code>formula-data</code>)</li> </ul> <p>You can either monitor the global power consumption of your machine, or monitor the power consumption of all Docker containers. As my project was using <a href="https://www.vagrantup.com/">Vagrant</a>, I decided to monitor the global power consumption of my computer. However, I recommend monitoring the Docker containers as it will reduce the “noise” created by your machine.</p> <p>To install PowerAPI, the <a href="http://powerapi.org/">PowerAPI documentation</a> will guide you. Once the MongoDB instance is created, the power meter can be deployed in two commands and a few minutes. Here’s a rapid recap, to measure the global power consumption:</p> <ul> <li> <p>Deploy the sensor: launch the first command to create the sensor Docker container.</p> <div class="gatsby-highlight" data-language="text"><pre class="language-text"><code class="language-text">docker run --privileged --net=host --name powerapi-sensor --privileged -td \ -v /sys:/sys \ -v /var/lib/docker/containers:/var/lib/docker/containers:ro \ -v /tmp/powerapi-sensor-reporting:/reporting powerapi/hwpc-sensor \ -n powerapi-sensor \ -r &quot;mongodb&quot; -U &quot;mongodb://ADDR&quot; -D &quot;power-data&quot; -C &quot;sensor-data&quot; \ -s &quot;rapl&quot; -o -e RAPL_ENERGY_PKG</code></pre></div> </li> <li> <p>Deploy the formula: launch the second command to create the formula Docker container.</p> <div class="gatsby-highlight" data-language="text"><pre class="language-text"><code class="language-text">docker run -td --net=host --name powerapi-formula powerapi/rapl-formula \ -s \ --input mongodb -u mongodb://ADDR -d power-data -c sensor-data \ --output mongodb -u mongodb://ADDR -d power-data -c formula-data</code></pre></div> </li> </ul> <p>The power meter is now ready, with two running Docker containers: the sensor and the formula.</p> <h3>Start measuring</h3> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/420ec04fa12690f229ae5574cf1c07a5/50383/powerapi-dockers.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 8.108108108108107%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAACCAYAAABYBvyLAAAACXBIWXMAAAsSAAALEgHS3X78AAAAgUlEQVQI1x2MyQ6CMBRF2TnEKMpoggbBoEhbploLAcL//9S1fbuTOxynPRaYkg5z0kOHHPJcYrlLLDcJ5TP07htDXGNNFXXd6UVsO7HPIS8f8F2GapPSz2EGVMAwXhvoSJDga0ZWokNhDiVlY9yQsD48MZid8iqw7YOkVmj5F3D8AVB4Oc0C6AOhAAAAAElFTkSuQmCC'); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="PowerAPI dockers running" title="PowerAPI dockers running" src="/static/420ec04fa12690f229ae5574cf1c07a5/50383/powerapi-dockers.png" srcset="/static/420ec04fa12690f229ae5574cf1c07a5/1d79a/powerapi-dockers.png 185w, /static/420ec04fa12690f229ae5574cf1c07a5/1efb2/powerapi-dockers.png 370w, /static/420ec04fa12690f229ae5574cf1c07a5/50383/powerapi-dockers.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <p>If both Dockers are running, the power estimation is stored in the database. To stop monitoring, stop the Dockers.</p> <h2>The results</h2> <p>Once PowerAPI was deployed:</p> <ul> <li>Step 1: I first started to monitor my computer energy consumption, to estimate what was the <strong>“noise” created by my machine</strong>.</li> <li>Step 2: I launched the <strong>vagrant</strong> to start my application</li> <li>Step 3: I ran the <strong>script</strong> 5 times in order to get average data.</li> </ul> <p>The formula had stored all my power consumption data in the database. Now, I needed a way to display and analyse it.</p> <h3>How to process data from PowerAPI</h3> <p>First, I decided to use <a href="https://docs.mongodb.com/charts/onprem/installation/">MongoDB Charts</a> to display my results.</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/5f7f2591bc0acddaa431b1c936d7d1a0/50383/mongodb-charts-example.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 45.4054054054054%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAJCAYAAAAywQxIAAAACXBIWXMAAAsTAAALEwEAmpwYAAABZUlEQVQoz11SW07DMBDM/U/EIRAffCGQKGqbR6tCcRLbsR1nmHGbqMLKyOPd9ezDqYwxMMMI7yxSnDBNATFGhEBwF08pbdA5Ppwf4ZxDZfqBxONtf8FXe0WeZzpnLMuCnHOBOPiJS3Ceb/7/SLRXo3XQev38wfN7S0FVdcuoi0KaE/poy16S0GYTu0mBPG9x8lUhBKiAFANGN2BSS2qX9iLAan/DiNp/4xqGkki2ITp4jqicKSZuo0dV2uHqOcfT+QLLijVXtbYwo4IkqAoN4ViZZiiuBBIW97SP5Jvg7zjA9D2892W4NngGBfTBliqG4EqLhuIDbUqiZLIrRogpPlQ4jriaHpaCet1AZ+A85zxvj1Me5T7HJS+lg9U/lVjOEPc18SEc4TkfCTZNg5bY7/foug51XeN4PKJtWxwOh+Jfd/nEz6cOVS5Z8vYr6LwKrpckIjFdVILdbrcJ13fBpj7i6eUDf2GWuXeIYyebAAAAAElFTkSuQmCC'); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="MongoDB Charts example" title="MongoDB Charts example" src="/static/5f7f2591bc0acddaa431b1c936d7d1a0/50383/mongodb-charts-example.png" srcset="/static/5f7f2591bc0acddaa431b1c936d7d1a0/1d79a/mongodb-charts-example.png 185w, /static/5f7f2591bc0acddaa431b1c936d7d1a0/1efb2/mongodb-charts-example.png 370w, /static/5f7f2591bc0acddaa431b1c936d7d1a0/50383/mongodb-charts-example.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <p>MongoDB provided me a simple interface to display the results written by the formula. I can for instance filter by date, to create a graph for a precise duration. On this chart, I displayed the energy consumption of my application during 1 minute.</p> <p>However, in order to extract more information from the results, I decided to extract the data as a CSV, which can be done:</p> <ul> <li>directly in the <a href="https://powerapi-ng.github.io/formula_cli.html#csv">formula parameters</a> by adding <code>--output csv</code> at the end of the command to launch the formula Docker</li> <li>with a MongoDB command</li> <li>via MongoDB Charts.</li> </ul> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/9b23ab7b1b9a3754c645ab64e32f2dd1/50383/mongodb-charts-results.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 42.16216216216216%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAICAYAAAD5nd/tAAAACXBIWXMAAAsSAAALEgHS3X78AAABr0lEQVQozzWRWY/TQBCE/f9/APCOhHgFBBIsy7HZHHbsia/xMUfYTVBOiSxRpDx+9HjFQ6lrqqtr3J7oer2y2+3Ybrccj0cO+z2bzWbge+Ghdzgc2O62Qw1a8DaNZrVaDeeAjWjn85nocrlQFDlpmuKcE2NDHMd0fY/WmjIvMMJztcAYQ1WWgoqf0xGF9KqipNENC+mH4Oh0OmGNxVlHb/phyEtwJ7w0DXlf47wbqvcebVq0bakfejpvxKNpbIfqSo6HI9FGVjHesvRLbvoZreuFe6yzJKbg3mR0oo2NwogWW9nGlmSmIrdazgXKVkw6NfySKD95vnRTfvtHXrefh6G1fxiCb/t4CAvhoXrnB/2XW/K2/TpcGHjQWtPJF4bAv0vedDfMXM6r7gO3NmbsFJ/MmJftewrf8M0mzF3JVDyJK8i95kX7jo/iqXzLyGVMZINmvyR6+nPibj7hRzZmmsaMsil3gpGakuaKdJFxr2bMVcb34FEJ4ywmVnMSlTLLEiZK5pIJetk/P0p4qbbW1FWNFjSVHmpdVdT1s1aWwdOgxdcIdND/Q/rFImf9uOYfFJ0x2OsyDI4AAAAASUVORK5CYII='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="Results on MongoDB Charts" title="Results on MongoDB Charts" src="/static/9b23ab7b1b9a3754c645ab64e32f2dd1/50383/mongodb-charts-results.png" srcset="/static/9b23ab7b1b9a3754c645ab64e32f2dd1/1d79a/mongodb-charts-results.png 185w, /static/9b23ab7b1b9a3754c645ab64e32f2dd1/1efb2/mongodb-charts-results.png 370w, /static/9b23ab7b1b9a3754c645ab64e32f2dd1/50383/mongodb-charts-results.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <p>Once I had the data, I started to analyze it.</p> <p>The results were given in Watts. When dealing with electric consumption, it is easier to use Watt-hours: for instance, you need 1 kWh to bake a cake in an oven during 1 hour. It means that your oven consumed 1000 Watts during 1 hour.</p> <table> <thead> <tr> <th></th> <th>Duration (s)</th> <th>Average Power (W)</th> <th>Consumption (Wh)</th> </tr> </thead> <tbody> <tr> <td><strong>Step 1</strong></td> <td>225.5</td> <td>0.81</td> <td>0.05</td> </tr> <tr> <td><strong>Step 2</strong></td> <td>225.5</td> <td>0.83</td> <td>0.05</td> </tr> <tr> <td><strong>Import</strong></td> <td>225.5</td> <td>5.47</td> <td>0.34</td> </tr> </tbody> </table> <p>The import lasted in average 225.5 seconds, about 3min45.</p> <ul> <li>In Step 1, my computer alone consumed on average 0.81 W, i.e. 0.05 Wh during the imports.</li> <li>In Step 2, my computer with the vagrant containing the application running used 0.83 W. Step 1 and Step 2 are almost equivalent in terms of Wh. In other words, running the application on Vagrant does not consume of lot of energy.</li> <li>On average, <strong>one import used 0.34 Wh</strong>. The electric consumption almost did not vary between imports.</li> </ul> <p>If I subtract the noise of the computer, an import alone consumes 0.29Wh. Imports are made 96 times a day, i.e. 35 064 times a year. It means the imports use each year around 10 kWh.</p> <p>A Danish association, Tomorrow, created an <a href="https://www.electricitymap.org/map">interactive map</a> to see in real time the countries' electricity production and its impact in terms of CO<sub>2</sub>. Based on this website and on an 24-hours average, in France, 10 kWh corresponds to <strong>3.5 kg of CO<sub>2</sub></strong> emitted, as our energy comes mostly from nuclear. In the United States, 10 kWh corresponds to 11 kg CO<sub>2</sub>.</p> <p>If the server running my application was behaving exactly like my computer, it would use around 7 kWh a year.</p> <p>To sum up, I estimated that the application was using 17 kWh each year, and <strong>60% of this energy consumption is made by the imports</strong>. The rest is the energy consumes by the server, in the hypothesis of a physical server.</p> <p>What can we usually do with 17 kWh? With 17 kWh, you can use your air-conditioning for 102 hours, cook 17 chickens in your oven or travel 8.5 km with a small electric car.</p> <p>It is quite a lot for the server of a small web application.</p> <h2>What can be done to reduce the impact of this application?</h2> <p>I spoke with the developer and the person which was updating the CSV file, and we decided to:</p> <ul> <li>take a quick action and rapidly stop importing the file outside work hours. Importing the file only between 9a.m. and 7p.m. on weekdays reduces the total consumption of the imports to 3 kWh a year, in other words a reduction of 41% of the electric consumption!</li> <li>in the long term, create a button in the application, which can only be seen by the person updating the CSV file. He would use it when he updates the file, a few times a day. It will reduce the emissions of 95%.</li> </ul> <h2>What about the cloud?</h2> <p>Thanks to PowerAPI, I was able to <strong>estimate the impact of a costly feature</strong> on a physical server. I provided figures to <strong>help the project team make decisions</strong> to reduce the carbon footprint of the application.</p> <p>However, this server runs on AWS, and PowerAPI cannot be used in a virtual environment in the cloud. In order to estimate the real impact, we need to find a methodology, using data from the cloud provider: for instance the amount of CPU used by the application.</p> <p>I will try to see the real impact in the cloud of this feature in an upcoming article!</p><![CDATA[Flask Debugging in VS Code with Hot-Reload 🔥]]>/2020/05/debug-flask-vscode//2020/05/debug-flask-vscode/Mon, 11 May 2020 00:00:00 GMT<style> .boxes { display: flex; align-items: flex-start; } .boxes .box {} .boxes.row { flex-flow: row nowrap; } .boxes.row > * { width: 10%; flex-grow: 1; } .boxes.row > *:not(:first-child) { margin-left: 16px; } .boxes.column { flex-flow: column nowrap; } .box { padding: 16px 16px 16px 44px; width: 100%; border-radius: 3px; border-width: 1px; border-style: solid; border-color: transparent; background: rgba(235, 236, 237, 0.3) none repeat scroll 0% 0%; position: relative; font-size: 0.9em; margin: 8px 0 1.45rem; } .box.blue { background: rgba(221, 235, 241, 0.3) none repeat scroll 0% 0%; } .box a { font-weight: normal; } .box ol:last-child { margin-bottom: 0; } .box li { margin-bottom: 0; } .box aside { position: absolute; top: 16px; left: 12px; font-size: 1.2em; } .wide-image-container { display: flex; justify-content: center; height: min(60vw, 600px); } .wide-image-container img { position: absolute; width: 1067px; } </style> <p>I love using a <strong>debugger</strong> when I code. It allows me to <strong>quickly understand</strong> why something does not work as intended, but also to get a <strong>faster and deeper understanding</strong> of code I did not write.</p> <p>Since I am so fond of using a <strong>debugger</strong>, when I started working on a Dockerized Flask application, my first online search was to find how to set one up for my application. But all the solutions I found had different flaws:</p> <ul> <li>❌ The application <strong>port</strong> would be <strong>changed every time</strong> I started the application with the debugger</li> <li>❌ Flask's wonderful <strong>hot-reload</strong> feature (the server restarting after saving changes to the code) was <strong>not supported</strong></li> <li>❌ The usage was <strong>clunky</strong>. To make the debugger work, I had to consistently add then remove multiple lines of work</li> </ul> <p>So, I decided to craft my own debugger setup to fix all those problems.</p> <p>Let's see how to setup a Dockerized Flask app with an <strong>efficient debugging flow</strong> 🎉!</p> <h2>🔧 Prerequisites</h2> <p>To follow this tutorial, you will only need the following installed:</p> <ul> <li><strong>VS Code</strong></li> <li><strong>Docker</strong></li> <li><strong>Docker Compose</strong></li> <li>The <a href="https://marketplace.visualstudio.com/items?itemName=ms-python.python"><strong>VS Code Python extension</strong></a></li> </ul> <h2>Step 1: Docker setup</h2> <p>To follow this tutorial, make sure you have a <strong>Docker</strong> configuration similar to this:</p> <div class="gatsby-highlight" data-language="yaml"><pre class="language-yaml"><code class="language-yaml"><span class="token comment"># docker-compose.yml</span> <span class="token key atrule">version</span><span class="token punctuation">:</span> <span class="token string">"3.4"</span> <span class="token key atrule">services</span><span class="token punctuation">:</span> <span class="token key atrule">flask-server</span><span class="token punctuation">:</span> <span class="token key atrule">image</span><span class="token punctuation">:</span> flask <span class="token key atrule">build</span><span class="token punctuation">:</span> <span class="token key atrule">context</span><span class="token punctuation">:</span> . <span class="token key atrule">dockerfile</span><span class="token punctuation">:</span> Dockerfile <span class="token key atrule">ports</span><span class="token punctuation">:</span> <span class="token punctuation">-</span> 5000<span class="token punctuation">:</span><span class="token number">5000</span> <span class="token key atrule">volumes</span><span class="token punctuation">:</span> <span class="token punctuation">-</span> .<span class="token punctuation">:</span>/app<span class="token punctuation">:</span>cached</code></pre></div> <div class="gatsby-highlight" data-language="dockerfile"><pre class="language-dockerfile"><code class="language-dockerfile"><span class="token comment"># Dockerfile</span> <span class="token keyword">FROM</span> python<span class="token punctuation">:</span>3.8 <span class="token keyword">EXPOSE</span> 5000 <span class="token comment"># Keeps Python from generating .pyc files in the container</span> <span class="token keyword">ENV</span> PYTHONDONTWRITEBYTECODE 1 <span class="token comment"># Turns off buffering for easier container logging</span> <span class="token keyword">ENV</span> PYTHONUNBUFFERED 1 <span class="token comment"># Install pip requirements</span> <span class="token keyword">ADD</span> requirements.txt . <span class="token keyword">RUN</span> python <span class="token punctuation">-</span>m pip install <span class="token punctuation">-</span>r requirements.txt <span class="token keyword">WORKDIR</span> /app <span class="token comment"># Switch to a non-root user, please refer to https://aka.ms/vscode-docker-python-user-rights</span> <span class="token keyword">RUN</span> useradd appuser &amp;&amp; chown <span class="token punctuation">-</span>R appuser /app <span class="token keyword">USER</span> appuser</code></pre></div> <p>Here we have setup a simple <code>flask-server</code> service that will run our Flask application inside a Docker container.</p> <h2>Step 2: Setup the debugger</h2> <h3>VS Code configuration</h3> <p>The only configuration you will need is adding or modifying the <code>.vscode/launch.json</code> file:</p> <div class="gatsby-highlight" data-language="json"><pre class="language-json"><code class="language-json"><span class="token comment">// .vscode/launch.json</span> <span class="token punctuation">{</span> <span class="token property">"configurations"</span><span class="token operator">:</span> <span class="token punctuation">[</span> <span class="token punctuation">{</span> <span class="token property">"name"</span><span class="token operator">:</span> <span class="token string">"Python: Remote Attach"</span><span class="token punctuation">,</span> <span class="token property">"type"</span><span class="token operator">:</span> <span class="token string">"python"</span><span class="token punctuation">,</span> <span class="token property">"request"</span><span class="token operator">:</span> <span class="token string">"attach"</span><span class="token punctuation">,</span> <span class="token property">"port"</span><span class="token operator">:</span> <span class="token number">10001</span><span class="token punctuation">,</span> <span class="token property">"host"</span><span class="token operator">:</span> <span class="token string">"localhost"</span><span class="token punctuation">,</span> <span class="token property">"pathMappings"</span><span class="token operator">:</span> <span class="token punctuation">[</span> <span class="token punctuation">{</span> <span class="token property">"localRoot"</span><span class="token operator">:</span> <span class="token string">"${workspaceFolder}"</span><span class="token punctuation">,</span> <span class="token property">"remoteRoot"</span><span class="token operator">:</span> <span class="token string">"/app"</span> <span class="token punctuation">}</span> <span class="token punctuation">]</span> <span class="token punctuation">}</span> <span class="token punctuation">]</span> <span class="token punctuation">}</span></code></pre></div> <div class="blue box"> <aside>💡</aside> <ul> <li> Be sure to properly setup the <code>pathMappings</code> property. This will be used to link the files being executed in the Docker container to the files in your machine. </li> <li> The <code>.vscode</code> folder stores all the project's VS Code configuration files. </li> </ul> </div> <h3>Install the debugpy Python module</h3> <p><a href="https://github.com/microsoft/debugpy"><code>debugpy</code></a> is a Python module that will allow you to spawn a debugger inside our Python code.</p> <p>To install it, make sure to add <code>debugpy</code> to your <code>requirements.txt</code>:</p> <div class="gatsby-highlight" data-language="diff"><pre class="language-diff"><code class="language-diff">flask==1.1.2 <span class="token inserted-sign inserted"><span class="token prefix inserted">+</span>debugpy</span></code></pre></div> <div class="blue box"> <aside>💡</aside> Not specifying the version (ie: not adding a line like this <code>debugpy==1.0.0</code>) will automatically install the latest version of the module. </div> <h3>Use debugpy to create a debug adapter instance</h3> <p>Create a <code>debugger.py</code> file in your application:</p> <div class="gatsby-highlight" data-language="python"><pre class="language-python"><code class="language-python"><span class="token comment"># debugger.py</span> <span class="token keyword">from</span> os <span class="token keyword">import</span> getenv <span class="token keyword">def</span> <span class="token function">initialize_flask_server_debugger_if_needed</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">:</span> <span class="token keyword">if</span> getenv<span class="token punctuation">(</span><span class="token string">"DEBUGGER"</span><span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token string">"True"</span><span class="token punctuation">:</span> <span class="token keyword">import</span> multiprocessing <span class="token keyword">if</span> multiprocessing<span class="token punctuation">.</span>current_process<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span>pid <span class="token operator">></span> <span class="token number">1</span><span class="token punctuation">:</span> <span class="token keyword">import</span> debugpy debugpy<span class="token punctuation">.</span>listen<span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token string">"0.0.0.0"</span><span class="token punctuation">,</span> <span class="token number">10001</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">"⏳ VS Code debugger can now be attached, press F5 in VS Code ⏳"</span><span class="token punctuation">,</span> flush<span class="token operator">=</span><span class="token boolean">True</span><span class="token punctuation">)</span> debugpy<span class="token punctuation">.</span>wait_for_client<span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">"🎉 VS Code debugger attached, enjoy debugging 🎉"</span><span class="token punctuation">,</span> flush<span class="token operator">=</span><span class="token boolean">True</span><span class="token punctuation">)</span></code></pre></div> <p>Let's explain what is happening here:</p> <h4>Debug adapter logic</h4> <div class="gatsby-highlight" data-language="python"><pre class="language-python"><code class="language-python">debugpy<span class="token punctuation">.</span>listen<span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token string">"0.0.0.0"</span><span class="token punctuation">,</span> <span class="token number">10001</span><span class="token punctuation">)</span><span class="token punctuation">)</span></code></pre></div> <p>This will start the debug adapter that will listen for a client connection at the <code>0.0.0.0:10001</code> interface.</p> <div class="gatsby-highlight" data-language="python"><pre class="language-python"><code class="language-python">debugpy<span class="token punctuation">.</span>wait_for_client<span class="token punctuation">(</span><span class="token punctuation">)</span></code></pre></div> <p>This line will block program execution until a client (in our case, the client will be the VS Code debugger) is attached.</p> <h4>More logic for a better experience</h4> <div class="gatsby-highlight" data-language="python"><pre class="language-python"><code class="language-python"><span class="token keyword">if</span> getenv<span class="token punctuation">(</span><span class="token string">"DEBUGGER"</span><span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token string">"True"</span><span class="token punctuation">:</span></code></pre></div> <p>We want the debugger to be spawned only if the <code>DEBUGGER</code> env variable is set to <code>True</code>. That way, we will still be able to run our application without it.</p> <div class="gatsby-highlight" data-language="python"><pre class="language-python"><code class="language-python"><span class="token keyword">if</span> multiprocessing<span class="token punctuation">.</span>current_process<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span>pid <span class="token operator">></span> <span class="token number">1</span><span class="token punctuation">:</span></code></pre></div> <p>In debug mode Flask uses a first process (with <code>pid==1</code>) to start child processes that handle connections. If the code below this line is executed by the main process, the debugging port is taken and subsequent child processes can't use the same port and are attributed a random port which prevents connections.</p> <h3>Instantiate the debugger</h3> <p>Given that your Flask application instance <code>app</code> is created in a file named <code>app.py</code>, call the <code>initialize_flask_server_debugger_if_needed</code> function inside it like this:</p> <div class="gatsby-highlight" data-language="python"><pre class="language-python"><code class="language-python"><span class="token comment"># app.py</span> <span class="token keyword">from</span> flask <span class="token keyword">import</span> Flask <span class="token keyword">from</span> debugger <span class="token keyword">import</span> initialize_flask_server_debugger_if_needed initialize_flask_server_debugger_if_needed<span class="token punctuation">(</span><span class="token punctuation">)</span> app <span class="token operator">=</span> Flask<span class="token punctuation">(</span>__name__<span class="token punctuation">)</span> <span class="token decorator annotation punctuation">@app<span class="token punctuation">.</span>route</span><span class="token punctuation">(</span><span class="token string">"/"</span><span class="token punctuation">)</span> <span class="token keyword">def</span> <span class="token function">home</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">:</span> <span class="token keyword">return</span> <span class="token string">"Hello, Flask!"</span></code></pre></div> <br /> <h3>Expose the debugging port</h3> <p>To be able to listen to the debug port of your Docker container, add the <code>10001</code> port to the list of exposed ports:</p> <div class="gatsby-highlight" data-language="yaml"><pre class="language-yaml"><code class="language-yaml"><span class="token comment"># docker-compose.yml</span> <span class="token punctuation">...</span> <span class="token key atrule">services</span><span class="token punctuation">:</span> <span class="token key atrule">flask-server</span><span class="token punctuation">:</span> <span class="token punctuation">...</span> <span class="token key atrule">ports</span><span class="token punctuation">:</span> <span class="token punctuation">-</span> 5000<span class="token punctuation">:</span><span class="token number">5000</span> <span class="token punctuation">-</span> 10001<span class="token punctuation">:</span><span class="token number">10001</span> <span class="token punctuation">...</span></code></pre></div> <br /> <h3>Launch your application with a debugger 🎉</h3> <p>Everything is set up, now we only need to start our application!</p> <p>Here are the <code>docker-compose</code> commands in a <code>Makefile</code> to launch your application with hot-reload using the <code>flask</code> executable.</p> <div class="gatsby-highlight" data-language="makefile"><pre class="language-makefile"><code class="language-makefile"><span class="token comment"># Makefile</span> <span class="token comment">## 🌶 flask and hot-reload</span> <span class="token symbol">flask</span><span class="token punctuation">:</span> docker-compose run --rm -e FLASK_APP<span class="token operator">=</span>app.py -e FLASK_ENV<span class="token operator">=</span>development --service-ports flask-server flask run --host 0.0.0.0 <span class="token symbol">flaskdebug</span><span class="token punctuation">:</span> docker-compose run --rm -e DEBUGGER<span class="token operator">=</span>True -e FLASK_APP<span class="token operator">=</span>app.py -e FLASK_ENV<span class="token operator">=</span>development --service-ports flask-server flask run --host 0.0.0.0</code></pre></div> <p>The only difference between the two commands is the <code>-e DEBUGGER=True</code> parameter. This will set the <code>DEBUGGER</code> env variable to <code>True</code> inside your container, allowing the debugger code to be executed.</p> <p>Here's a GIF to show the debugger in action:</p> <div class="wide-image-container"> <img alt="VS Code debugger in action" src="/00209783c8f9f39be8c374fe449b95da/how-to-use-debugger.gif"> </div> <p>To summarize, here are the steps to follow:</p> <ul> <li>In a terminal, run <code>make flaskdebug</code></li> <li>When prompted <code>⏳ VS Code debugger can now be attached, press F5 in VS Code ⏳</code>, press <code>F5</code></li> <li>Add a breakpoint to the line you want to debug by clicking left of its number</li> <li>Call the corresponding route (<code>localhost:5000/hello/flask</code> in the GIF)</li> <li>The debugger stops the code at your breakpoint, happy debugging 🎉</li> </ul> <h3>Hot-reload in action 🔥</h3> <p>Now that it's easy to use the debugger, let's see how the application <strong>hot-reload</strong> feature is supported.</p> <div class="wide-image-container"> <img alt="Debugger with hot-reload" src="/066b565795ed2ac193ed832ceb8ccd67/hot-reload.gif" style> </div> <p>As you can see in the GIF, after I saved modifications to a file:</p> <ul> <li>The application restarts</li> <li>The debugger is detached</li> <li>I am prompted <code>⏳ VS Code debugger can now be attached, press F5 in VS Code ⏳</code> again</li> <li>After I pressed <code>F5</code>, the application is now running with the new changes and the debugger is attached</li> </ul> <hr> <h2>Tips &#x26; tricks</h2> <h3>Use the Debug Console to code on the fly</h3> <p>Whenever you are stopped at a breakpoint in your code, VS Code's <code>Debug Console</code> acts as a Python interactive console with the full current context of your code.</p> <div class="wide-image-container"> <img alt="VS Code Debug Console example" src="/8d5fbc73020aed5c5458b65e1a24f86e/debug-console.gif" style> </div> <p>Here you can see that I was able to do the following:</p> <ul> <li>Import <code>date</code> from the <code>datetime</code> module</li> <li>Re-assign the <code>name</code> value to <code>date(3000, 1, 1)</code></li> <li>The route response is correctly updated with the modifications</li> </ul> <div class="blue box"> <aside>💡</aside> The <code>Debug Console</code> is a very useful tool to quickly test edge cases and find fixes! </div> <h3>Running the app with gunicorn</h3> <p>To use <a href="https://gunicorn.org/"><code>gunicorn</code></a> instead of <code>flask</code> to run the app, use these commands:</p> <div class="gatsby-highlight" data-language="makefile"><pre class="language-makefile"><code class="language-makefile"><span class="token comment">## 🦄 gunicorn and hot-reload</span> <span class="token symbol">gunicorn</span><span class="token punctuation">:</span> <span class="token symbol"> docker-compose run --rm --service-ports flask-server gunicorn --reload --bind 0.0.0.0</span><span class="token punctuation">:</span>5000 app<span class="token punctuation">:</span>app <span class="token symbol">gunicorndebug</span><span class="token punctuation">:</span> docker-compose run --rm -e DEBUGGER<span class="token operator">=</span>True --service-ports flask-server gunicorn --reload --bind 0.0.0.0<span class="token punctuation">:</span>5000 --timeout 3600 app<span class="token punctuation">:</span>app</code></pre></div> <br /> <h3>Running the app as a top-level script</h3> <p>If you need to run your app as a <a href="https://docs.python.org/3/library/__main__.html">top-level script</a>, ie:</p> <ul> <li>The app is run with <code>python app.py</code></li> <li> <div class="gatsby-highlight" data-language="python"><pre class="language-python"><code class="language-python"><span class="token comment"># app.py</span> <span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span> <span class="token keyword">if</span> __name__ <span class="token operator">==</span> <span class="token string">"__main__"</span><span class="token punctuation">:</span> app<span class="token punctuation">.</span>run<span class="token punctuation">(</span><span class="token string">"0.0.0.0"</span><span class="token punctuation">,</span> debug<span class="token operator">=</span><span class="token boolean">True</span><span class="token punctuation">)</span></code></pre></div> </li> </ul> <p>Be sure to make this modification:</p> <div class="gatsby-highlight" data-language="python"><pre class="language-python"><code class="language-python"><span class="token keyword">if</span> __name__ <span class="token operator">==</span> <span class="token string">"__main__"</span><span class="token punctuation">:</span> <span class="token keyword">from</span> debugger <span class="token keyword">import</span> initialize_flask_server_debugger_if_needed initialize_flask_server_debugger_if_needed<span class="token punctuation">(</span><span class="token punctuation">)</span> app<span class="token punctuation">.</span>run<span class="token punctuation">(</span><span class="token string">"0.0.0.0"</span><span class="token punctuation">,</span> debug<span class="token operator">=</span><span class="token boolean">True</span><span class="token punctuation">)</span></code></pre></div> <p>Thanks to this, if you want to use the <code>app</code> instance in another file:</p> <div class="gatsby-highlight" data-language="python"><pre class="language-python"><code class="language-python"><span class="token comment"># another-python-file.py</span> <span class="token keyword">from</span> app <span class="token keyword">import</span> app</code></pre></div> <p>The code initiating the debugger will not be executed when the import is made.</p> <h2>Wrap-up 🌯</h2> <p>I showed you how to resolve the three problems presented I encountered trying to setup a debugging flow in a Dockerized Flask application:</p> <ul> <li>✔️ The application <strong>port does not change</strong></li> <li>✔️ Flask's wonderful <strong>hot-reload</strong> feature is <strong>supported</strong></li> <li>✔️ The debugger is <strong>easy to use</strong></li> </ul> <p>I created a <a href="https://github.com/adriencaccia/vscode-flask-debug">repository</a> containing a minimal application with everything shown in the article.</p> <p>I hope that this article helped you setup a (better!) debugging flow of your Dockerized Flask application 💪.</p> <hr> <div class="blue box"> <aside>☝️</aside> <strong>You have a question?</strong> No problem, feel free to either contact me: <ul> <li>📧 by <a href="mailto:adrienc@theodo.fr">email</a></li> <li>🐦 on <a href="https://twitter.com/TheLorkis">twitter</a></li> <li>🐙 by leaving an issue or a pull request on the <a href="https://github.com/adriencaccia/vscode-flask-debug">tutorial repository</a></li> <li>💬 by leaving a comment below</li> </ul> I will be very happy to discuss this tutorial with you 😃 </div><![CDATA[Stop losing data when writing Django migrations !]]>/2020/05/django-migrations-without-losing-data//2020/05/django-migrations-without-losing-data/Thu, 07 May 2020 00:00:00 GMT<style> .gatsby-highlight { font-size: 0.7rem; } a { font-size: 0.8rem; } code { font-family: "SFMono-Regular", Consolas, "Liberation Mono", Menlo, Courier, monospace; line-height: normal; background: rgba(135,131,120,0.15); color: #EB5757; border-radius: 3px; padding: 0.2em 0.4em; font-size: 0.7rem; } </style> <p>Saying that database structure is important is sort of an obvious statement. That is of course if you decided to use a structured database technology. But in that case, you want your database structure to be the closest to your functional and business needs, and the tightest in order to be able to rely on that structure.</p> <p>However, when I was working on Django projects, I sometimes found it painful to change the data structure when I needed to. However, I discovered a very useful tool offered by Django migrations : the <code>RunPython</code> command.</p> <h2>Simple migration</h2> <p>Let’s take a simple example and assume that we have a single model called <code>Order</code> inside an <code>orders</code> application.</p> <p>We would then have the following <code>models.py</code> file:</p> <div class="gatsby-highlight" data-language="python"><pre class="language-python"><code class="language-python"><span class="token keyword">from</span> django<span class="token punctuation">.</span>db <span class="token keyword">import</span> models <span class="token keyword">class</span> <span class="token class-name">Order</span><span class="token punctuation">(</span>models<span class="token punctuation">.</span>Model<span class="token punctuation">)</span><span class="token punctuation">:</span> reference <span class="token operator">=</span> models<span class="token punctuation">.</span>CharField<span class="token punctuation">(</span>max_length<span class="token operator">=</span><span class="token number">8</span><span class="token punctuation">)</span> amount <span class="token operator">=</span> models<span class="token punctuation">.</span>DecimalField<span class="token punctuation">(</span>max_digits<span class="token operator">=</span><span class="token number">8</span><span class="token punctuation">,</span> decimal_places<span class="token operator">=</span><span class="token number">2</span><span class="token punctuation">)</span> creation_date <span class="token operator">=</span> models<span class="token punctuation">.</span>DateField<span class="token punctuation">(</span>auto_now<span class="token operator">=</span><span class="token boolean">False</span><span class="token punctuation">,</span> auto_now_add<span class="token operator">=</span><span class="token boolean">False</span><span class="token punctuation">)</span> due_date <span class="token operator">=</span> models<span class="token punctuation">.</span>DateField<span class="token punctuation">(</span>auto_now<span class="token operator">=</span><span class="token boolean">False</span><span class="token punctuation">,</span> auto_now_add<span class="token operator">=</span><span class="token boolean">False</span><span class="token punctuation">)</span> customer_name <span class="token operator">=</span> models<span class="token punctuation">.</span>CharField<span class="token punctuation">(</span>max_length<span class="token operator">=</span><span class="token number">50</span><span class="token punctuation">)</span> customer_address <span class="token operator">=</span> models<span class="token punctuation">.</span>CharField<span class="token punctuation">(</span>max_length<span class="token operator">=</span><span class="token number">50</span><span class="token punctuation">)</span> customer_city <span class="token operator">=</span> models<span class="token punctuation">.</span>CharField<span class="token punctuation">(</span>max_length<span class="token operator">=</span><span class="token number">50</span><span class="token punctuation">)</span> customer_zip_code <span class="token operator">=</span> models<span class="token punctuation">.</span>CharField<span class="token punctuation">(</span>max_length<span class="token operator">=</span><span class="token number">50</span><span class="token punctuation">)</span> <span class="token keyword">def</span> <span class="token function">__str__</span><span class="token punctuation">(</span>self<span class="token punctuation">)</span><span class="token punctuation">:</span> self<span class="token punctuation">.</span>reference</code></pre></div> <p>Running <code>python manage.py makemigrations</code> will produce the following migrations file, called <code>0001_initial.py</code> :</p> <div class="gatsby-highlight" data-language="python"><pre class="language-python"><code class="language-python"><span class="token keyword">from</span> django<span class="token punctuation">.</span>db <span class="token keyword">import</span> migrations<span class="token punctuation">,</span> models <span class="token keyword">class</span> <span class="token class-name">Migration</span><span class="token punctuation">(</span>migrations<span class="token punctuation">.</span>Migration<span class="token punctuation">)</span><span class="token punctuation">:</span> initial <span class="token operator">=</span> <span class="token boolean">True</span> dependencies <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token punctuation">]</span> operations <span class="token operator">=</span> <span class="token punctuation">[</span> migrations<span class="token punctuation">.</span>CreateModel<span class="token punctuation">(</span> name<span class="token operator">=</span><span class="token string">"Order"</span><span class="token punctuation">,</span> fields<span class="token operator">=</span><span class="token punctuation">[</span> <span class="token punctuation">(</span> <span class="token string">"id"</span><span class="token punctuation">,</span> models<span class="token punctuation">.</span>AutoField<span class="token punctuation">(</span> auto_created<span class="token operator">=</span><span class="token boolean">True</span><span class="token punctuation">,</span> primary_key<span class="token operator">=</span><span class="token boolean">True</span><span class="token punctuation">,</span> serialize<span class="token operator">=</span><span class="token boolean">False</span><span class="token punctuation">,</span> verbose_name<span class="token operator">=</span><span class="token string">"ID"</span><span class="token punctuation">,</span> <span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token string">"reference"</span><span class="token punctuation">,</span> models<span class="token punctuation">.</span>CharField<span class="token punctuation">(</span>max_length<span class="token operator">=</span><span class="token number">8</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token string">"amount"</span><span class="token punctuation">,</span> models<span class="token punctuation">.</span>DecimalField<span class="token punctuation">(</span>decimal_places<span class="token operator">=</span><span class="token number">2</span><span class="token punctuation">,</span> max_digits<span class="token operator">=</span><span class="token number">8</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token string">"creation_date"</span><span class="token punctuation">,</span> models<span class="token punctuation">.</span>DateField<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token string">"due_date"</span><span class="token punctuation">,</span> models<span class="token punctuation">.</span>DateField<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token string">"customer_name"</span><span class="token punctuation">,</span> models<span class="token punctuation">.</span>CharField<span class="token punctuation">(</span>max_length<span class="token operator">=</span><span class="token number">50</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token string">"customer_address"</span><span class="token punctuation">,</span> models<span class="token punctuation">.</span>CharField<span class="token punctuation">(</span>max_length<span class="token operator">=</span><span class="token number">50</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token string">"customer_city"</span><span class="token punctuation">,</span> models<span class="token punctuation">.</span>CharField<span class="token punctuation">(</span>max_length<span class="token operator">=</span><span class="token number">50</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token string">"customer_zip_code"</span><span class="token punctuation">,</span> models<span class="token punctuation">.</span>CharField<span class="token punctuation">(</span>max_length<span class="token operator">=</span><span class="token number">50</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">]</span></code></pre></div> <p>Let's break down what's happening in this migrations file. The migration operation is represented as a python class, with 3 attributes:</p> <ul> <li><code>initial</code> states that this migration is the first of its Django app;</li> <li><code>dependencies</code> list all the migrations that need to be applied before this one can be applied. Here it is empty since the migration is the first of its Django app;</li> <li><code>operations</code> is the most important part. It is a list of actions that Django will apply to the database. Here the database was empty, so we created the model from scratch with the <code>CreateModel</code> method.</li> </ul> <p>The <code>python manage.py makemigrations</code> performs well when the operations to perform are simple enough. Renaming a model, or renaming a field will be treated as equivalent database operations for example.</p> <p>After applying the migration, let's create a few entities in the database.</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/e73338b6e7de8daa742b062534a3cba4/50383/runpython-step1-large.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 19.459459459459456%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAECAYAAACOXx+WAAAACXBIWXMAAAsSAAALEgHS3X78AAAAs0lEQVQY0z1P2Q6DIBDk/z+yaZqqSJAWEQ8Qpsy29mGTPeZaZYyB1hrbtmHfd8QYZV6WBcdxYAkB4zhiGAbMrSfmwjrnBMub97Ps1NM63LTFeZ4opQiYJill1Fq/Bo1AInti8g9Lwb7v/4G4U3fj8LAv5JSaSMK6RkyTk57FpCR572VmalbOueEmdF33v3OnjA/Q74Da1OlA0jwH6UtLyDdsI64tARNfn9RaxIyi1loJwvsHKsgz97JSs6sAAAAASUVORK5CYII='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="Orders after running the first migration" title="Orders after running the first migration" src="/static/e73338b6e7de8daa742b062534a3cba4/50383/runpython-step1-large.png" srcset="/static/e73338b6e7de8daa742b062534a3cba4/1d79a/runpython-step1-large.png 185w, /static/e73338b6e7de8daa742b062534a3cba4/1efb2/runpython-step1-large.png 370w, /static/e73338b6e7de8daa742b062534a3cba4/50383/runpython-step1-large.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <h2>Complex structure migration</h2> <p>Let's start again from our previous example and observe something that you may already have noticed. The 4 fields <code>customer_name</code>, <code>customer_address</code>, <code>customer_city</code>, and <code>customer_zip_code</code> are functionally attached to the <code>Order</code> entity, while they represent another physical entity, a customer. And that is fine as long as there is no duplication. But now let's imagine that for business purposes, you need to abstract a <code>Customer</code> entity to which the <code>Order</code> entities are linked.</p> <p>Your <code>models.py</code> file would then be:</p> <div class="gatsby-highlight" data-language="python"><pre class="language-python"><code class="language-python"><span class="token keyword">from</span> django<span class="token punctuation">.</span>db <span class="token keyword">import</span> models <span class="token keyword">class</span> <span class="token class-name">Customer</span><span class="token punctuation">(</span>models<span class="token punctuation">.</span>Model<span class="token punctuation">)</span><span class="token punctuation">:</span> name <span class="token operator">=</span> models<span class="token punctuation">.</span>CharField<span class="token punctuation">(</span>max_length<span class="token operator">=</span><span class="token number">50</span><span class="token punctuation">)</span> address <span class="token operator">=</span> models<span class="token punctuation">.</span>CharField<span class="token punctuation">(</span>max_length<span class="token operator">=</span><span class="token number">50</span><span class="token punctuation">)</span> city <span class="token operator">=</span> models<span class="token punctuation">.</span>CharField<span class="token punctuation">(</span>max_length<span class="token operator">=</span><span class="token number">50</span><span class="token punctuation">)</span> zip_code <span class="token operator">=</span> models<span class="token punctuation">.</span>CharField<span class="token punctuation">(</span>max_length<span class="token operator">=</span><span class="token number">50</span><span class="token punctuation">)</span> <span class="token keyword">def</span> <span class="token function">__str__</span><span class="token punctuation">(</span>self<span class="token punctuation">)</span><span class="token punctuation">:</span> <span class="token keyword">return</span> self<span class="token punctuation">.</span>name <span class="token keyword">class</span> <span class="token class-name">Order</span><span class="token punctuation">(</span>models<span class="token punctuation">.</span>Model<span class="token punctuation">)</span><span class="token punctuation">:</span> reference <span class="token operator">=</span> models<span class="token punctuation">.</span>CharField<span class="token punctuation">(</span>max_length<span class="token operator">=</span><span class="token number">8</span><span class="token punctuation">)</span> amount <span class="token operator">=</span> models<span class="token punctuation">.</span>DecimalField<span class="token punctuation">(</span>max_digits<span class="token operator">=</span><span class="token number">8</span><span class="token punctuation">,</span> decimal_places<span class="token operator">=</span><span class="token number">2</span><span class="token punctuation">)</span> creation_date <span class="token operator">=</span> models<span class="token punctuation">.</span>DateField<span class="token punctuation">(</span>auto_now<span class="token operator">=</span><span class="token boolean">False</span><span class="token punctuation">,</span> auto_now_add<span class="token operator">=</span><span class="token boolean">False</span><span class="token punctuation">)</span> due_date <span class="token operator">=</span> models<span class="token punctuation">.</span>DateField<span class="token punctuation">(</span>auto_now<span class="token operator">=</span><span class="token boolean">False</span><span class="token punctuation">,</span> auto_now_add<span class="token operator">=</span><span class="token boolean">False</span><span class="token punctuation">)</span> customer <span class="token operator">=</span> models<span class="token punctuation">.</span>ForeignKey<span class="token punctuation">(</span>Customer<span class="token punctuation">,</span> on_delete<span class="token operator">=</span>models<span class="token punctuation">.</span>PROTECT<span class="token punctuation">)</span> <span class="token keyword">def</span> <span class="token function">__str__</span><span class="token punctuation">(</span>self<span class="token punctuation">)</span><span class="token punctuation">:</span> <span class="token keyword">return</span> self<span class="token punctuation">.</span>reference</code></pre></div> <p>This configuration now fits our functional needs. Let's try and generate the migration !</p> <div class="gatsby-highlight" data-language="bash"><pre class="language-bash"><code class="language-bash"><span class="token operator">></span> You are trying to <span class="token function">add</span> a non-nullable field <span class="token string">'customer'</span> to order without a default<span class="token punctuation">;</span> we can't <span class="token keyword">do</span> that <span class="token punctuation">(</span>the database needs something to populate existing rows<span class="token punctuation">)</span>. Please <span class="token keyword">select</span> a fix: <span class="token number">1</span><span class="token punctuation">)</span> Provide a one-off default now <span class="token punctuation">(</span>will be <span class="token builtin class-name">set</span> on all existing rows with a null value <span class="token keyword">for</span> this <span class="token function">column</span><span class="token punctuation">)</span> <span class="token number">2</span><span class="token punctuation">)</span> Quit, and <span class="token builtin class-name">let</span> me <span class="token function">add</span> a default <span class="token keyword">in</span> models.py</code></pre></div> <p>There is a problem however. Django recognizes that we are trying to create a <code>customer</code> field on the <code>Order</code> model, which can't be <code>None</code> but is not defined. So it wants to add a non-null value to all the existing entries in the database, and asks us if we want to provide it now or want to set it in the <code>Order</code> model.</p> <p>However, none of these options satisfy us. We don't want our orders to have customers that are either <code>None</code> or a default customer, which would lose all the existing data.</p> <p>Granted, we could also execute this migration, then manually set the <code>customer</code> attribute of all the <code>Order</code> model, but then this forces you to execute this same action on all of your environments. That would also make the migration quite difficult to rollback.</p> <p>Fortunately, Django comes with an built-in solution to deal with this limitation.</p> <h2>Generating a custom migration</h2> <p>First of, let's start by generating an empty migration that we will then edit.</p> <div class="gatsby-highlight" data-language="bash"><pre class="language-bash"><code class="language-bash"><span class="token operator">></span> python manage.py makemigrations orders --empty</code></pre></div> <p>The generated migration file will look like this</p> <div class="gatsby-highlight" data-language="python"><pre class="language-python"><code class="language-python"><span class="token keyword">from</span> django<span class="token punctuation">.</span>db <span class="token keyword">import</span> migrations <span class="token keyword">class</span> <span class="token class-name">Migration</span><span class="token punctuation">(</span>migrations<span class="token punctuation">.</span>Migration<span class="token punctuation">)</span><span class="token punctuation">:</span> dependencies <span class="token operator">=</span> <span class="token punctuation">[</span> <span class="token punctuation">(</span><span class="token string">'orders'</span><span class="token punctuation">,</span> <span class="token string">'0001_initial'</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">]</span> operations <span class="token operator">=</span> <span class="token punctuation">[</span> <span class="token punctuation">]</span></code></pre></div> <p>We then need to build our custom migration. It needs to have 6 steps :</p> <ul> <li>Create the new <code>Customer</code> model in the database, with all the necessary fields.</li> <li>Create a nullable <code>customer</code> foreign key on <code>Order</code></li> <li>Set the <code>Order</code> fields to transfer as nullable</li> <li>Transfer the data from the <code>Order</code> model to the <code>Customer</code> model.</li> <li>Set the new <code>customer</code> field on <code>Order</code> as non-nullable</li> <li>Remove the old fields from the <code>Order</code> model.</li> </ul> <p>The Django documentation explains all the database operations needed.</p> <p>Let's write this migration :</p> <div class="gatsby-highlight" data-language="python"><pre class="language-python"><code class="language-python"><span class="token keyword">from</span> django<span class="token punctuation">.</span>db <span class="token keyword">import</span> migrations<span class="token punctuation">,</span> models <span class="token keyword">import</span> django<span class="token punctuation">.</span>db<span class="token punctuation">.</span>models<span class="token punctuation">.</span>deletion <span class="token keyword">class</span> <span class="token class-name">Migration</span><span class="token punctuation">(</span>migrations<span class="token punctuation">.</span>Migration<span class="token punctuation">)</span><span class="token punctuation">:</span> dependencies <span class="token operator">=</span> <span class="token punctuation">[</span> <span class="token punctuation">(</span><span class="token string">"orders"</span><span class="token punctuation">,</span> <span class="token string">"0001_initial"</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">]</span> operations <span class="token operator">=</span> <span class="token punctuation">[</span> <span class="token comment"># step 1: add the new Customer model</span> migrations<span class="token punctuation">.</span>CreateModel<span class="token punctuation">(</span> name<span class="token operator">=</span><span class="token string">"Customer"</span><span class="token punctuation">,</span> fields<span class="token operator">=</span><span class="token punctuation">[</span> <span class="token punctuation">(</span> <span class="token string">"id"</span><span class="token punctuation">,</span> models<span class="token punctuation">.</span>AutoField<span class="token punctuation">(</span> auto_created<span class="token operator">=</span><span class="token boolean">True</span><span class="token punctuation">,</span> primary_key<span class="token operator">=</span><span class="token boolean">True</span><span class="token punctuation">,</span> serialize<span class="token operator">=</span><span class="token boolean">False</span><span class="token punctuation">,</span> verbose_name<span class="token operator">=</span><span class="token string">"ID"</span><span class="token punctuation">,</span> <span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token string">"name"</span><span class="token punctuation">,</span> models<span class="token punctuation">.</span>CharField<span class="token punctuation">(</span>max_length<span class="token operator">=</span><span class="token number">50</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token string">"address"</span><span class="token punctuation">,</span> models<span class="token punctuation">.</span>CharField<span class="token punctuation">(</span>max_length<span class="token operator">=</span><span class="token number">50</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token string">"city"</span><span class="token punctuation">,</span> models<span class="token punctuation">.</span>CharField<span class="token punctuation">(</span>max_length<span class="token operator">=</span><span class="token number">50</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token string">"zip_code"</span><span class="token punctuation">,</span> models<span class="token punctuation">.</span>CharField<span class="token punctuation">(</span>max_length<span class="token operator">=</span><span class="token number">50</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token comment"># step 2: add the nullable foreign key field `customer` to Order</span> migrations<span class="token punctuation">.</span>AddField<span class="token punctuation">(</span> model_name<span class="token operator">=</span><span class="token string">"order"</span><span class="token punctuation">,</span> name<span class="token operator">=</span><span class="token string">"customer"</span><span class="token punctuation">,</span> field<span class="token operator">=</span>models<span class="token punctuation">.</span>ForeignKey<span class="token punctuation">(</span> null<span class="token operator">=</span><span class="token boolean">True</span><span class="token punctuation">,</span> on_delete<span class="token operator">=</span>django<span class="token punctuation">.</span>db<span class="token punctuation">.</span>models<span class="token punctuation">.</span>deletion<span class="token punctuation">.</span>PROTECT<span class="token punctuation">,</span> to<span class="token operator">=</span><span class="token string">"orders.Customer"</span><span class="token punctuation">,</span> <span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token comment"># step 3: set the order fields as nullable</span> migrations<span class="token punctuation">.</span>AlterField<span class="token punctuation">(</span> model_name<span class="token operator">=</span><span class="token string">"order"</span><span class="token punctuation">,</span> name<span class="token operator">=</span><span class="token string">"customer_address"</span><span class="token punctuation">,</span> field<span class="token operator">=</span>models<span class="token punctuation">.</span>CharField<span class="token punctuation">(</span>null<span class="token operator">=</span><span class="token boolean">True</span><span class="token punctuation">,</span> max_length<span class="token operator">=</span><span class="token number">50</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">)</span><span class="token punctuation">,</span> migrations<span class="token punctuation">.</span>AlterField<span class="token punctuation">(</span> model_name<span class="token operator">=</span><span class="token string">"order"</span><span class="token punctuation">,</span> name<span class="token operator">=</span><span class="token string">"customer_city"</span><span class="token punctuation">,</span> field<span class="token operator">=</span>models<span class="token punctuation">.</span>CharField<span class="token punctuation">(</span>null<span class="token operator">=</span><span class="token boolean">True</span><span class="token punctuation">,</span> max_length<span class="token operator">=</span><span class="token number">50</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">)</span><span class="token punctuation">,</span> migrations<span class="token punctuation">.</span>AlterField<span class="token punctuation">(</span> model_name<span class="token operator">=</span><span class="token string">"order"</span><span class="token punctuation">,</span> name<span class="token operator">=</span><span class="token string">"customer_name"</span><span class="token punctuation">,</span> field<span class="token operator">=</span>models<span class="token punctuation">.</span>CharField<span class="token punctuation">(</span>null<span class="token operator">=</span><span class="token boolean">True</span><span class="token punctuation">,</span> max_length<span class="token operator">=</span><span class="token number">50</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">)</span><span class="token punctuation">,</span> migrations<span class="token punctuation">.</span>AlterField<span class="token punctuation">(</span> model_name<span class="token operator">=</span><span class="token string">"order"</span><span class="token punctuation">,</span> name<span class="token operator">=</span><span class="token string">"customer_zip_code"</span><span class="token punctuation">,</span> field<span class="token operator">=</span>models<span class="token punctuation">.</span>CharField<span class="token punctuation">(</span>null<span class="token operator">=</span><span class="token boolean">True</span><span class="token punctuation">,</span> max_length<span class="token operator">=</span><span class="token number">50</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token comment"># step 4: transfer data from Order to Customer</span> <span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span> <span class="token comment"># step 5: set the `customer` field as non-nullable</span> migrations<span class="token punctuation">.</span>AlterField<span class="token punctuation">(</span> model_name<span class="token operator">=</span><span class="token string">"order"</span><span class="token punctuation">,</span> name<span class="token operator">=</span><span class="token string">"customer"</span><span class="token punctuation">,</span> field<span class="token operator">=</span>models<span class="token punctuation">.</span>ForeignKey<span class="token punctuation">(</span> null<span class="token operator">=</span><span class="token boolean">False</span><span class="token punctuation">,</span> on_delete<span class="token operator">=</span>django<span class="token punctuation">.</span>db<span class="token punctuation">.</span>models<span class="token punctuation">.</span>deletion<span class="token punctuation">.</span>PROTECT<span class="token punctuation">,</span> to<span class="token operator">=</span><span class="token string">"orders.Customer"</span><span class="token punctuation">,</span> <span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token comment"># step 6: remove the old Order fields</span> migrations<span class="token punctuation">.</span>RemoveField<span class="token punctuation">(</span>model_name<span class="token operator">=</span><span class="token string">"order"</span><span class="token punctuation">,</span> name<span class="token operator">=</span><span class="token string">"customer_address"</span><span class="token punctuation">,</span><span class="token punctuation">)</span><span class="token punctuation">,</span> migrations<span class="token punctuation">.</span>RemoveField<span class="token punctuation">(</span>model_name<span class="token operator">=</span><span class="token string">"order"</span><span class="token punctuation">,</span> name<span class="token operator">=</span><span class="token string">"customer_city"</span><span class="token punctuation">,</span><span class="token punctuation">)</span><span class="token punctuation">,</span> migrations<span class="token punctuation">.</span>RemoveField<span class="token punctuation">(</span>model_name<span class="token operator">=</span><span class="token string">"order"</span><span class="token punctuation">,</span> name<span class="token operator">=</span><span class="token string">"customer_name"</span><span class="token punctuation">,</span><span class="token punctuation">)</span><span class="token punctuation">,</span> migrations<span class="token punctuation">.</span>RemoveField<span class="token punctuation">(</span>model_name<span class="token operator">=</span><span class="token string">"order"</span><span class="token punctuation">,</span> name<span class="token operator">=</span><span class="token string">"customer_zip_code"</span><span class="token punctuation">,</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">]</span></code></pre></div> <p>In this migration, step 2 and 3 basically loosen the data structure, preparing it for the data transfer. Then step 5 and 6 tighten it again !</p> <p>Let's now dive in step 4, where the magic happens !</p> <h1>Write the data transfer function</h1> <p>In order to perform this operation, we are going to use the <code>RunPython</code> migration operation. What it basically does is execute a python function using the ORM.</p> <p>Its syntax is the following :</p> <div class="gatsby-highlight" data-language="python"><pre class="language-python"><code class="language-python"><span class="token comment"># step 4: transfer data from Order to Customer</span> migrations<span class="token punctuation">.</span>RunPython<span class="token punctuation">(</span>order_to_customer<span class="token punctuation">,</span> reverse_code<span class="token operator">=</span>customer_to_order<span class="token punctuation">)</span></code></pre></div> <p>In this step of the migration, we specify two functions :</p> <ul> <li><code>order_to_customer</code> will be run when running the migration ;</li> <li><code>customer_to_order</code> will be run when reversing the migration.</li> </ul> <div class="gatsby-highlight" data-language="python"><pre class="language-python"><code class="language-python"><span class="token keyword">def</span> <span class="token function">order_to_customer</span><span class="token punctuation">(</span>apps<span class="token punctuation">,</span> schema_editor<span class="token punctuation">)</span><span class="token punctuation">:</span> Order <span class="token operator">=</span> apps<span class="token punctuation">.</span>get_model<span class="token punctuation">(</span><span class="token string">"orders"</span><span class="token punctuation">,</span> <span class="token string">"Order"</span><span class="token punctuation">)</span> Customer <span class="token operator">=</span> apps<span class="token punctuation">.</span>get_model<span class="token punctuation">(</span><span class="token string">"orders"</span><span class="token punctuation">,</span> <span class="token string">"Customer"</span><span class="token punctuation">)</span> <span class="token keyword">for</span> order <span class="token keyword">in</span> Order<span class="token punctuation">.</span>objects<span class="token punctuation">.</span><span class="token builtin">all</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">:</span> customer<span class="token punctuation">,</span> _ <span class="token operator">=</span> Customer<span class="token punctuation">.</span>objects<span class="token punctuation">.</span>get_or_create<span class="token punctuation">(</span> name<span class="token operator">=</span>order<span class="token punctuation">.</span>customer_name<span class="token punctuation">,</span> address<span class="token operator">=</span>order<span class="token punctuation">.</span>customer_address<span class="token punctuation">,</span> city<span class="token operator">=</span>order<span class="token punctuation">.</span>customer_city<span class="token punctuation">,</span> zip_code<span class="token operator">=</span>order<span class="token punctuation">.</span>customer_zip_code<span class="token punctuation">,</span> <span class="token punctuation">)</span> order<span class="token punctuation">.</span>customer <span class="token operator">=</span> customer order<span class="token punctuation">.</span>save<span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token keyword">def</span> <span class="token function">customer_to_order</span><span class="token punctuation">(</span>apps<span class="token punctuation">,</span> schema_editor<span class="token punctuation">)</span><span class="token punctuation">:</span> Order <span class="token operator">=</span> apps<span class="token punctuation">.</span>get_model<span class="token punctuation">(</span><span class="token string">"orders"</span><span class="token punctuation">,</span> <span class="token string">"Order"</span><span class="token punctuation">)</span> <span class="token keyword">for</span> order <span class="token keyword">in</span> Order<span class="token punctuation">.</span>objects<span class="token punctuation">.</span><span class="token builtin">all</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">:</span> order<span class="token punctuation">.</span>customer_name <span class="token operator">=</span> order<span class="token punctuation">.</span>customer<span class="token punctuation">.</span>name order<span class="token punctuation">.</span>customer_address <span class="token operator">=</span> order<span class="token punctuation">.</span>customer<span class="token punctuation">.</span>address order<span class="token punctuation">.</span>customer_city <span class="token operator">=</span> order<span class="token punctuation">.</span>customer<span class="token punctuation">.</span>city order<span class="token punctuation">.</span>customer_zip_code <span class="token operator">=</span> order<span class="token punctuation">.</span>customer<span class="token punctuation">.</span>zip_code order<span class="token punctuation">.</span>save<span class="token punctuation">(</span><span class="token punctuation">)</span></code></pre></div> <p>These functions look a lot like what you would write to manually transfer the data from one model to the other. However there is one trick. It isn't possible to import the models normally with <code>from orders.models import Order, Customer</code>. Instead, we need to use a versioned model passed to the migration function by Django.</p> <h2>Running the migration</h2> <p>After running this migration via <code>python manage.py migrate</code>, we can check that the migration has created the <code>Customer</code> instances and linked them to the correct <code>Orders</code>.</p> <h3>Customers</h3> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/169af9a79c4ea5eb13288389376f3ca0/50383/runpython-step2-large.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 20.54054054054054%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAECAYAAACOXx+WAAAACXBIWXMAAAsSAAALEgHS3X78AAAApklEQVQY001Q7Q6DIBDz/R9y2ZJtDGSIosiH2lEyjD96d+m19EK3LAv6voeUEiGEinVda5+mCZ/Ce+8RY6z8FeSstVBKnb4ulPLUBjepsW3biX3fMY4jhBDQX13N5No+54zjOGCHAcaYOpPvmH5XBg89IKdUjQQNTH+933DOIZUdEVv/axhKHWdyXSpFWgdRwAvaFez8Dq11faBdcNWQ80Uzz/O5/wHIPTZ5Yq2LAAAAAABJRU5ErkJggg=='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="Customers after running the second migration" title="Customers after running the second migration" src="/static/169af9a79c4ea5eb13288389376f3ca0/50383/runpython-step2-large.png" srcset="/static/169af9a79c4ea5eb13288389376f3ca0/1d79a/runpython-step2-large.png 185w, /static/169af9a79c4ea5eb13288389376f3ca0/1efb2/runpython-step2-large.png 370w, /static/169af9a79c4ea5eb13288389376f3ca0/50383/runpython-step2-large.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <h3>Orders</h3> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/16ba59a4339f7860f9a65d6072aedb35/50383/runpython-step3-large.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 16.216216216216214%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAADCAYAAACTWi8uAAAACXBIWXMAAAsSAAALEgHS3X78AAAAeklEQVQI122Oyw7DIAwE+f+f7KXUCAFNIMTmpS1JL1WTw0jWemSvYt7hvUNKCa1WiDCY7xERpBhBxoDnXEq5OKrMhQkLnn7B6x0hpWL0jn7DGAN7ztBaw7lviSP7dZTMqxRWPGzAuuXza51N/zny1hpy3kBEJ9bai/8BYwjojpDLnD4AAAAASUVORK5CYII='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="Orders after running the second migration" title="Orders after running the second migration" src="/static/16ba59a4339f7860f9a65d6072aedb35/50383/runpython-step3-large.png" srcset="/static/16ba59a4339f7860f9a65d6072aedb35/1d79a/runpython-step3-large.png 185w, /static/16ba59a4339f7860f9a65d6072aedb35/1efb2/runpython-step3-large.png 370w, /static/16ba59a4339f7860f9a65d6072aedb35/50383/runpython-step3-large.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <p><em>Et voilà !</em> This migration is now operational. It can also be easily rolled back with <code>python manage.py migrate orders &#x3C;previous_migration_id></code>. The data will then be transferred back to the <code>Order</code> model.</p> <h2>Conclusion</h2> <p>You now have a way to stop worrying about losing data when migrating your database with Django! The Django ORM is a great tool and although it does the job most of the time, understanding what happens under the hood is a great way of making your life easier when dealing with migrations.</p> <h2>Sources</h2> <ul> <li> <p>Django documentation on migrations:</p> <ul> <li><a href="https://docs.djangoproject.com/en/3.0/howto/writing-migrations/">https://docs.djangoproject.com/en/3.0/howto/writing-migrations/</a></li> <li><a href="https://docs.djangoproject.com/en/3.0/ref/migration-operations/">https://docs.djangoproject.com/en/3.0/ref/migration-operations/</a></li> </ul> </li> <li>Link to the Github repo with the example <a href="https://github.com/fargito/django-runpyton">https://github.com/fargito/django-runpyton</a></li> </ul><![CDATA[How to Remain Agile with DynamoDB]]>/2020/05/remain-agile-dynamodb//2020/05/remain-agile-dynamodb/Tue, 05 May 2020 00:00:00 GMT<p>Amazon DynamoDB is built to deliver single-digit millisecond performance at any scale. It is built to store Terabytes of data. It is built to support Amazon's Cyber Monday traffic.</p> <p>However, this scalability comes with an overhead. No matter what resource you look up, you will be constantly reminded that "<strong>You Must Know Your Access Patterns In Advance</strong>". Amazon Marketplace had been running for years before they migrated to DynamoDB so they were well equipped to know how they needed to access their data and fully reap the rewards of a NoSQL setup.</p> <p>In contrast, a new startup with an exciting new product wants to build an MVP quickly. They don't know how their app will grow over time. They want to be agile and react to their users to develop their product accordingly. They can't know all their access patterns in advance.</p> <p>This no doubt poses an issue, but not an insurmountable one.</p> <p>Being able to map your access patterns in as much detail as possible will translate to a smooth DynamoDB adventure. However, building a startup is about disrupting the market so you can't be afraid of a little turbulence along to way to reach your goal.</p> <p>If you use DynamoDB from the start you will never have to worry about the scalability of your product. If your app gets a massive social media spike, you won't have to scramble to prevent your site going down. And reaching that scale is what all startups aspire to so how can we build towards this while limiting the turbulence?</p> <p>Check out my full article on our <a href="https://medium.com/serverless-transformation/how-to-remain-agile-with-dynamodb-eca44ff9817">Serverless Transformation Blog</a>.</p><![CDATA[Why You Need a Framework When Doing Serverless]]>/2020/04/serverless-framework-partnership//2020/04/serverless-framework-partnership/Mon, 27 Apr 2020 00:00:00 GMT<p>We are proud to have been selected as a Serverless Framework official partner. It rewards our investment in the technology and reflects our strong belief that this framework is currently the best in the market.</p> <h2>Using a Framework When Going Serverless is a Must</h2> <p>Recently, one of our clients’ technical advisors shared with us the challenges that he had encountered when testing serverless a few years ago. As he started to use AWS Lambda functions, he quickly lacked visibility of the infrastructure design and struggled to figure out which version of the code was running in production. Fortunately, these issues are now solved by frameworks for serverless applications.</p> <p>At Theodo, we began leveraging serverless functions a few years ago for projects requiring asynchronous workloads. To avoid the aforementioned pitfalls, we looked into tools that would allow us to be more efficient in managing our serverless projects. The most popular tool was, and remains today, the <a href="https://serverless.com">Serverless Framework</a>.</p> <p>The main advantage of the Serverless Framework is to allow developers to write to one single repository containing:</p> <ul> <li>The code of the business logic run by their serverless functions,</li> <li> <p>All the files that describe their application’s infrastructure, including:</p> <ul> <li>Database services like DynamoDB,</li> <li>Authentication services like Cognito or Auth0,</li> <li>Notification services like AWS Simple Notification Service or Twilio,</li> </ul> </li> <li>The scripts they need to easily deploy their application in one command to any environment,</li> <li>Tools allowing developers to emulate services like DynamoDB in their local development environment. Other things that the Serverless Framework achieves is the deployment of a new stack with the whole infrastructure in a single command, allowing to run automated tests consistently on a ephemeral — hence cheap — production-like environment. This further stresses the productivity gains and cost reductions observed by companies embracing serverless.</li> </ul> <h2>Why We Chose to Work With the Serverless Framework</h2> <h3>Its Large Community Guarantees its Relevance</h3> <p>The suite of serverless services offered by cloud providers is constantly expanding. Therefore, the value of a serverless application framework lies in the ability of its community to stay current.</p> <p>The Serverless Framework has the largest community among its competitors, and it provides many plugins that gradually add support for new services as they are released. You can read about a few of our favorite tools for this framework in this article.</p> <h3>Its Independence From Cloud Providers</h3> <p>Other frameworks, like AWS’ own Serverless Application Model (or SAM), are open-sourced and maintained by cloud providers themselves — offering early support for their newest features and services. The purpose of some third-party frameworks, however, is to enable developers to deploy their serverless applications to any cloud provider. As the concerns about vendor lock-in grow, the appeal of using a cloud-agnostic framework becomes stronger. It allows you to move your infrastructure to another cloud provider if necessary. </p> <p>The development of a reliable and powerful framework is a daunting and ambitious task, but we believe in the purpose of the Serverless Framework and we enjoy using it daily on many of our projects. We contribute to the community, for example with <a href="https://github.com/Theodo-UK/sls-dev-tools">sls-dev-tools</a>, our open-source project that gives developers visibility on their serverless application directly from the command line.</p> <p>With close to sixteen million downloads and hundreds of thousands of weekly deployments, Serverless Framework has become the benchmark in the serverless ecosystem — and this is only the beginning. </p> <p>Want to know more about how Serverless can help your company? I'd be happy to help. <a href="https://www.theodo.com/request-a-consultation">Please reach out here!</a></p><![CDATA[Reduce redux boilerplate with redux-toolkit]]>/2020/01/reduce-redux-boilerplate//2020/01/reduce-redux-boilerplate/Mon, 20 Apr 2020 00:00:00 GMT<blockquote> <p>You're looking for an easier way to write your reducer, aren't you ?</p> </blockquote> <p>When writing plain redux, you have to define an <code>action</code>, an <code>actionCreator</code> and a <code>reducer</code> to update the store of your app. The reducer is in most cases a big switch to handle all the actions dispatched by actionCreators in your app. This leads to a lot of boilerplate and make the code more difficult to understand. There are many librairies out there aiming to reduce this boilerplate.</p> <p>At Theodo, we used <strong>typesafe-actions</strong> to reduce redux boilerplate. The redux team recently recommended <strong>@redux-toolkit</strong> as the <a href="https://redux.js.org/introduction/getting-started#redux-toolkit">"official recommended approach for writing Redux logic"</a>. Thus, I tested it and wrote this article to show you the advantages of such librairies through a simple example.</p> <hr> <h3><em>The example</em></h3> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 300px; " > <span class="gatsby-resp-image-background-image" style="padding-bottom: 106.48648648648648%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAVCAYAAABG1c6oAAAACXBIWXMAABYlAAAWJQFJUiTwAAAEt0lEQVQ4y62UaWyUVRSGJ5FfJmqi0ZCwtJ3u7XSlreybYLBFkJZiF5SCFIJiLEURUJGCBjGCGgkgSglgqCAGECht6Ea36T5tp9ONdma60UKny0yX6bSdebzfhynxj/EHJ3lzzrnLue+5596jQMhgRyvdRbfI/v0chTcuo047jb6+CoeYm5icZNJuZ8JhZ8z+GDZJiznLmA2zoYEBUz92vRr6jSjGbDYelmVSeySBD8Jm8P2B3Zz8NJFLPx6gbnAIi90hY1gK4JDgYED4HT2dlP96iIyU96nJu03vlSQs2gwRUCwaLr7MkQhv1s8PIH7lAjL2RvHXd0lk3G9lYGSUjr4B2npNdDwy0WUZpqKpkYKvtpG3ZQ45e5agSwnkVsJ06qsqUDgsffTfPMGfiUvY/8YcPoleyenNyymr1VDy0ER6cyuZLUauNRv4Vl3F8SodaZXV/JT0Lmc3+nEqejZXN75I1qYXMNaWozCPWClp6eazfXtIWB/Ovvi1ZKRdoKFvkF8qtWQYu7igqeNMcQXZ+nZOl2rI7ezhxo2bnEpcwZl1L3Mt/hXOrX6W1poyFBZTB3dSP+aHves4nxTBndxctHoD/eoCbh9JofvSWfK++ZI/9u1i+MpF8o8dpTP1Z0w1tajrdRyKXc0cTxe8lG5UVWlQmNrrSU8O4eFhZ/pTN3HhYArnN0dhXuRMZtQKzi7w5beFPlxaGkjq8le5GBVO24ZlFG+L5urRw6RvjWX/jq18nrABQ3MDihGzmer8EnrrtBQcTOK483OUKp8hLcSJ0DVRzFsVwdyINcxfvZbFQs9bu57ty+aj9ZjGdddpXI1cSqsokrZUzZB5EIXdOopZ5F6bfp2vY99ii78rJ+e685HrS0x3c8df5UuAv7+MwAB/3L28CXSayYmgWSTP82Pv64soKCiksrIKi8WC4oHRQNzyxawLC2Z7dBQ7VErSPJ9np2o2PgGBhPh4E6xSEeznR5CvL4EqPxYGBXAyYCZfeM9gQ3g42z/cRVLybgwGAwpdXR1vLl7IsZ2JJL8Tg6uHJ94eHgT4+hAm2IUJHSp0qJ+KEBEsROggwdZb2E6eXqiCggkJDcPLV4WmuhpFS2Mj0a8tZdWCuXi6uaF0ccFV6YpSqcRJ2LOdnf+FWU5OOAm4CFvp4iyvd3f3YOasWVRUiIfd0dbGewmbiI6O5u2YGGJjY4mLi5MRHx8/Zcu+rJ/MS2tj/tkTGRlJoyCnsNulFvB0xCH+uUIytFotRUVFtAm2Op2Ompoa8Uir0Ov1U3ZpaSlNTU2UlZXR0tJCc3Oz7GdlZZGZmSmvk0QOaDQaKSwslAeLi4u5d+8e2dnZcvDKykrqROHu3r0r+3l5eVPj0kE5OTnkit+l0WgeB5RoPvWUjeItSqx0ujrqRPpSSrW1tTJbiYmUmsRSrVaTn59PtXgeJSUlNDQ0UF5eLmcn7ZEZToqiDFpG6eu3YBa9rn9ANNWhUUZGRhgaGpL16OgT32QyYRbf1Wq1ymOStokmPSk6uxxwZGySR+YJylutMu5327Da/v819Pb2ThVtfHwchdVmp6d/nPsPrOh7xug02Rgdk05zyHfyX5BkYGCArq4u2tvbmZiY4G9WyvJk4C6NOQAAAABJRU5ErkJggg=='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="Screenshot de l&#39;application développée" title="Screenshot de l&#39;application développée" src="/static/a5bab4c4dcc13438f42cee7d9750f520/5a46d/app-screenshot.png" srcset="/static/a5bab4c4dcc13438f42cee7d9750f520/1d79a/app-screenshot.png 185w, /static/a5bab4c4dcc13438f42cee7d9750f520/5a46d/app-screenshot.png 300w" sizes="(max-width: 300px) 100vw, 300px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </span></p> <blockquote> <p>The example consist in changing the owner of a car :</p> <ul> <li>There is one car and 2 possible owners ("JOHN" or "JANE").</li> <li>When you click on one of the owner, the car owner in the redux store is changed to match the owner clicked.</li> </ul> </blockquote> <hr> <!-- Thus will be needed : - an item (the car), - an action (updateOwner), - and a reducer. --> <h2>Plain redux (28 lines)</h2> <p>With plain redux as on the code below, there are :</p> <ul> <li>an <code>action</code> (UPDATE_OWNER),</li> <li>an <code>actionCreator</code> (updateOwner)</li> <li>a <code>reducer</code> (itemReducer) to handle all the actions (<a href="https://codesandbox.io/s/base-redux-example-pr3i2?file=/src/store/store.ts">codesandbox</a>).</li> </ul> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <span class="gatsby-resp-image-background-image" style="padding-bottom: 89.1891891891892%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAASCAYAAABb0P4QAAAACXBIWXMAABYlAAAWJQFJUiTwAAACSUlEQVQ4y41UWXLiMBTkGCHgRftqYQcClaq5/7V6WjbOeCaQyUeXBLKfennPu0YoOHOCNYWrgzeBa0GrDdwpwhQHGSz2xw4vxGvT47XdQmxWgV0rFab4gRJvXDN+5ROuccAljRjzBeP0BhM9XI7wp4TeGrRKzTj28gvmglonSBWhlUXSDlpqZG0RuffCwZCtcYmIkN5AJwfh7Ser107gcMeuUwZWFyRPxIhcMjQZyiHDUnK8DFDOoqU1R6FxlAYNV6k1FNlKo9ErzXO9MtRwtiCnN0yUWyhxON/QBxbyATZlOHqoXaBcjzZQDfcpeQxDQORqeaHQHjWPXU9p0nmU8xk+kVFKyJkvMaBDJ+8hLMYfVlDanv+/EPv7+adkWW+VC916sG+qJ4vBy0NLgUcBPAxFmGo6ZcUJdrohUa6olzCgjqH0xqPj/kvRTRDbs7mg4IuhTEil4HI9w1F2GQuu3L8Tdd8I+SOmlEwW0sG5Ammr8ZFpKiYnZ3RywWrBjwq2wmDw7xjLiFMhuxxYRN3Nll9k/aegx6EXEMojhTdYtkUr/mZ0+Mev77DTPi7VWSQOEZfLBM9RC2kJo6/znQt7jk2fl/4zjhalAYZkIs8DYZnDYQ2lFrSOD5g099tWwsqymX8/kb1NWbHr64PejijhyvYpc9s88+yRzO3/uzoRtbH3XYd92266fp2KP5+qH6VcGfaSsyj9Ag56TbjhbHaWAXGtwdX5bflhWC6T34ei6J25w1rOtjHoXYaIJ36u8px8DaEWrl7WJn+G39q1+sAXw0uCAAAAAElFTkSuQmCC'); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="base-redux-example-screenshot.png" title="base-redux-example-screenshot.png" src="/static/122fdf9fe58a5cf851a70716f1b34684/50383/base-redux-example-screenshot.png" srcset="/static/122fdf9fe58a5cf851a70716f1b34684/1d79a/base-redux-example-screenshot.png 185w, /static/122fdf9fe58a5cf851a70716f1b34684/1efb2/base-redux-example-screenshot.png 370w, /static/122fdf9fe58a5cf851a70716f1b34684/50383/base-redux-example-screenshot.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </span></p> <p>The first simplification external librairies provide is to <strong>get rid of the action and avoid the switch in the reducer function.</strong></p> <p>Both typesafe-actions and redux/toolkit give you two functions to simplify action and reducer : <code>createAction and createReducer</code></p> <blockquote> <ul> <li>We do not need to keep the UPDATE_OWNER action type anymore in an variable to handle it in the reducer.</li> <li>The switch disapears</li> <li>Code is easier to read and understand</li> </ul> </blockquote> <hr> <h2>Typesafe action (21 lines ~ -25%)</h2> <ul> <li>With createAction and createReducer functions there is no action constant UPDATE_OWNER and no verbose switch in our reducer anymore : the reducer reacts directly to the actionCreator updateOwner (<a href="https://codesandbox.io/s/redux-with-typesafe-actions-3y5nv?file=/src/store/store.ts">codesandbox</a>)</li> </ul> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <span class="gatsby-resp-image-background-image" style="padding-bottom: 73.5135135135135%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAPCAYAAADkmO9VAAAACXBIWXMAABYlAAAWJQFJUiTwAAACCUlEQVQ4y4VTWVLjQBTLNQix3fvuJQuBKu5/Lo1eExhSkJkPVXtpy3qSejdogxw2pHhE9BEtFOS4QrmIuFSEY4afM/ajwtMw4emgsB9u4LPnSd9hNxiLJV8xl1eUmPFeF1xLQ4sNW1px3Y5IOSHwPvJ5WDNsibA5wqSAgzJ32InCms5o+UJlCS+54Zgqzqlg7esJhR/neUbbNkQhzB+Ek3Vd1UF9UzgaB2tLh7ERiWN7Hzh+4MrrkOF8gXYJhtDRdwtMil3RB+HftRN6X5HzRtDHlFEKCWUc/kCVIyZCU7Fv/MGcuGaMNvwg64ST9TD0Li8cpy0IZab5NFw2yRjjdIOEoG9Qv5J1QhPoBUcbSSybn+VF90N9w32S4tlvZJ1QuQDnKlKasbImSysQ1RMtMPMFup7gGJTPlROsiGzBZMNjQk11mp27XN7wcrni/fWMmcSieDCfcBi0hfgteET2RXjg5sA0q09M1/eKWN4bQnOCxlSXhe9bQmYHpWoPPbQhcYODIVlbT33c58l0RR8wGEXd7Vrwb4VUIB/6lKiAfWS5ZUSfZ6TKY1gSHFsgHkobRPUjsk7o2C9NleKNJLr/TLJDddwnbH6kfneWlfNQJhAssQ7QPE6T5XhMcvQssJwO+mz4U7HjkbIvhbE21oanwLden8BeusDahApdNugsZW9ILL2NBcN/CP8AGfunsUx9hWsAAAAASUVORK5CYII='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="example-with-type-safe-actions.png" title="example-with-type-safe-actions.png" src="/static/940671e7ef6d238214e058e45c19f312/50383/example-with-type-safe-actions.png" srcset="/static/940671e7ef6d238214e058e45c19f312/1d79a/example-with-type-safe-actions.png 185w, /static/940671e7ef6d238214e058e45c19f312/1efb2/example-with-type-safe-actions.png 370w, /static/940671e7ef6d238214e058e45c19f312/50383/example-with-type-safe-actions.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </span></p> <ul> <li>Type-safe action provides also a clearer way to write the reducer using the chain API (<a href="https://codesandbox.io/s/redux-with-typesafe-actions-2-0l0jp?file=/src/store/store.ts"><em>codesandbox</em></a>)</li> </ul> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <span class="gatsby-resp-image-background-image" style="padding-bottom: 67.56756756756756%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAABYlAAAWJQFJUiTwAAAB8UlEQVQ4y5VT2XIaQQzkLxJjdpn73sMsjl/y/7/VaQ1gUoSiKg9d0s7Ro25pdyWe0PIZJSTMqaKlD9hQUNYZeauIa8FBGexHReiO9xuOuu/domDX0hlT+YUcE77KhHMuqLFiiQu2iaQ5w4eGkIg5wdcEWyIc8UjWCeu1wswKtyREFSeJPV+RU0KeGuqykDBDxwCTA47W/UPWCZ2rEBjjUXyE9wGJMfuEFDKCzzCOlYUKHTzClOFY5aDtc8neV5TyQWknRMorfD1Gj4NlFVwXKFbrGmXW2AmP3LuT3auTtZ2h1LqckOcVsc7d/IMckgtXvD/BjeBxbaco72h9Jxoe/PgffBNaW+lbQWuJciNG4zCmGaptXW7MCdOUEBgtpQt8mWFiwRh5rp6g8sJ7/iqZ5m9fv7GeP7FtMwkitJOmZFg2w0Q2QZAKlAuoLWNZKh/hrDKXmVU89+2hJOJjW9beFO0cDG2IhPMemlHzUYEQGu5brhvv+tkjFQ3a3QnlQ4jkZSE+KAuXGxtUkTi8lgPvM8cq5I63v/6Y/fikKaMMKJMfh/FymBv74Yg3+ZZI/LzmfQJeNOXioc24DHfh8MYuaWSTNI1WaaJ3FYG/pGUTLlOgX3Z7pzRlOcqx7KLz3ZeRJqvYoGi42CANkTjoy6XhBf4AeQeMydhCMOkAAAAASUVORK5CYII='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="example-with-type-safe-actions-2.png" title="example-with-type-safe-actions-2.png" src="/static/be9e70a3104d82bfaadf9f6e1fac5651/50383/example-with-type-safe-actions-2.png" srcset="/static/be9e70a3104d82bfaadf9f6e1fac5651/1d79a/example-with-type-safe-actions-2.png 185w, /static/be9e70a3104d82bfaadf9f6e1fac5651/1efb2/example-with-type-safe-actions-2.png 370w, /static/be9e70a3104d82bfaadf9f6e1fac5651/50383/example-with-type-safe-actions-2.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </span></p> <ul> <li>You can even combine actions easily and/or chain action handlers:</li> </ul> <div class="gatsby-highlight" data-language="text"><pre class="language-text"><code class="language-text">const counterReducer = createReducer(0) .handleAction([add, increment], (state, action) =&gt; state + (action.type === &#39;ADD&#39; ? action.payload : 1) ) .handleAction(add, (state=&gt; action) =&gt; ...);</code></pre></div> <blockquote> <p>The code is clearer and easier to understand</p> </blockquote> <div style="background-color: #E5E8E8; padding: 20px 20px 20px 40px; margin-bottom: 20px;"> Remarks : <br> - @reduxjs/toolkit provides also a `createAction` function but used in a slightly different manner <a href='https://redux-starter-kit.js.org/api/createaction/#createaction' target='_blank'>see here</a>. <br> - @reduxjs/toolkit `createReducer` function is used as typesafe-action </div> <hr> <h2>@redux/toolkit (18 lines ~ -35%)</h2> <ul> <li>@reduxjs/toolkit goes further and provides a function createSlice, allowing actions and reducers to be created in the same function. It's very powerfull, give it a try <a href="https://codesandbox.io/s/redux-with-redux-toolkit-iiord?file=/src/store/store.ts"><em>codesandbox</em></a>.</li> </ul> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <span class="gatsby-resp-image-background-image" style="padding-bottom: 70.27027027027026%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAABYlAAAWJQFJUiTwAAAB0klEQVQ4y5WTWZLcIBBE+xq9SGKnQEiylgn7w/e/VjpBtqcd4Z6xPzJQhOBRWVlcOmMxyo5cFRO+5QlfUkYIC3LeMH4tcKNABYfBOdwHjXuvz1VpPAbTvh/KNF06bTDJgTEdDfiWCnZqlRElzVjKBpkTxm1G2WfoGBq4t/aEqXdYA/as0DtWUWUDihdEF1GCUAT5xP8B1mU4n2FSgC8CI+EV0BFGi7HAi0fgAesdfPS4KotOZgy0rkJu1m60e+tUW58t/9IJjBnLemDeFpR1JnhEzAVSFvg04aEJpp4reaWLsh4DNRIoyxt62joB7BMvq6BWVdVw6v6BLiGNqMH4sqIc37FsO459ggsBHS33BCoGoBmEto7fru2v1XR/rZAN79ohj8QwxHlYBmLZBsvUlY9IWTBNCaUk5DpCvOB5VP4AGh6uQMump2nFWDIkpwY3NV32s278zOpvy5qHag9rz17d+j9qFSof2gjchnMcrj9DuA/vQVyfAqkHX1rW7JE2tGcTmy4c3gDh0DrOouIGX8eKEjoJ1EewBrRREPwE4duNnu83sX8pwvH5GT69yDnMfN/CkAzBnwJrysoEVknrrdI6fxxil9CHQhHEtHvu+xfLPwDv0InaAwvaLQAAAABJRU5ErkJggg=='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="example-with-redux-toolkit.png" title="example-with-redux-toolkit.png" src="/static/df455bbf07422632171c5b852a20a20b/50383/example-with-redux-toolkit.png" srcset="/static/df455bbf07422632171c5b852a20a20b/1d79a/example-with-redux-toolkit.png 185w, /static/df455bbf07422632171c5b852a20a20b/1efb2/example-with-redux-toolkit.png 370w, /static/df455bbf07422632171c5b852a20a20b/50383/example-with-redux-toolkit.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </span></p> <ul> <li>Moreover <em>Redux toolkit</em> is shipped with <strong>ImmerJS</strong>, "a tiny package that allows you to work with immutable state in a more convenient way".</li> </ul> <p>No need to duplicate the state with spread operators everywhere: You can directly update the state and ImmerJs will take into account the changes and create a new state from the precedent state + the changes you made. Have a look again to the reducer function 👇🏽</p> <div class="gatsby-highlight" data-language="text"><pre class="language-text"><code class="language-text">const { actions, reducer: itemReducer } = createSlice({ name: &quot;items&quot;, initialState: initialItemState, reducers: { updateOwner: (state: ItemState, { payload: { user, item } }: UpdateOwnerAction) =&gt; { // This long code with many ... is replaced by one line thks to ImmerJS // return { // ...state, // [item.id]: { ...state[item.id], owner: user } // } state[item.id].owner = user; } } });</code></pre></div> <blockquote> <p>This makes reducer and actions easier to understand. Goodbye <code>"desctructuration ..."</code> 😍.</p> </blockquote> <blockquote> <p>⚠️ Redux teams recommends to read <a href="https://immerjs.github.io/immer/docs/pitfalls">ImmerJs pitfalls</a> before using it, so read it 😉</p> </blockquote> <hr> <ul> <li>In the end of March 2020, the redux toolkit team added 2 api functions (createEntityAdapter &#x26; createAsyncThunk) to simplify reducer logic.</li> </ul> <p>In the last example I use the createEntityAdapter function : the entityAdapter "generates a set of prebuilt reducers and selectors for performing CRUD operations on a normalized state structure."</p> <p>After normalizing my itemState and creating the itemsAdapter, I can simplify my reducer function to use the generic entityAdapter.updateOne to handle the ownerUpdate action (<a href="https://codesandbox.io/s/redux-with-redux-toolkit-2-gub7q?file=/src/store/store.ts"><em>check full code here</em></a>).</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <span class="gatsby-resp-image-background-image" style="padding-bottom: 62.70270270270271%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAANCAYAAACpUE5eAAAACXBIWXMAABYlAAAWJQFJUiTwAAAB1UlEQVQ4y31TW27cMBDbM8TW+y1Z8i7SIil6/7OxlDYbFEGyH7QE2+JwONRFK43X8obkC0YI+FMqbiEiuohrPNGPhN4TgjJQQhIGxhhYr6GNgnF6QUiBfd9xMdrgvf0lYcVJovdU8Ctm/I6Ja8BrHBgu4G0UnGfB7XpFyRmpeYTkEMt9lVLeCZVSCCbCsWryBplws7L1MKHA+AxFtYoFtLUQal8Ht5cN27Z9rvOdEAKX2XK0GYE/t2TRs4cloSSJqydMrLCpwZWxijwOfoelcBJmWzFbz/TFWxagJ0fUkPsLhHGwJLP5oG/hKeFSOB8lV9RaMW4V/Uo1jt7EiJQLYmLLLCak+vTpKeH8yQfPSXYUkjrHltl+oJpI3wKJp89iv3u3z4Mf+x8JZ7tjjEUaGJ2oNa4kHCRP/Db3J/HwSXyQfkuoedgYi22fk5oQC4KQXwzf/lO2/6CShDMmHooeKcmwqqlYrnWq14SaXfC7UWrt92ceToXFNUYnrfjU6DAY1oMRsiStM58s2jmo2XaxbhF+VfoAg83YuLrCPZG9hWN0pGUOywkdG29CRuaVVCw+Q/x0ynMIvXIgreOoHeNoOBpjdHS0cUPrvM+d1+880VpD5rV7hn9o6VL1fUndpAAAAABJRU5ErkJggg=='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="example with @/redux-toolkit and createEntityAdapter function.png" title="example with @/redux-toolkit and createEntityAdapter function.png" src="/static/bcb19df1eabbdab22f36fcfed9b4b100/50383/example-with-redux-toolkit-2.png" srcset="/static/bcb19df1eabbdab22f36fcfed9b4b100/1d79a/example-with-redux-toolkit-2.png 185w, /static/bcb19df1eabbdab22f36fcfed9b4b100/1efb2/example-with-redux-toolkit-2.png 370w, /static/bcb19df1eabbdab22f36fcfed9b4b100/50383/example-with-redux-toolkit-2.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </span></p> <p>This gives you a very fast way to write your reducer logic.</p> <p>Have a look at redux-toolkit doc to fully understand the <a href="https://redux-starter-kit.js.org/api/createEntityAdapter">createEntityAdapter API</a></p> <h1>Conclusion :</h1> <p><em>Reasons to use typesafe-actions</em></p> <ul> <li> <p>If you have a lot of actions to handle and one reducer has to react to several actions then typesafe-actions chaining will be right for you</p> <p>⚠️ Typesafe-actions maintainer is currently updating their API to simpllify the syntax. If you want to contribute. ⚠️</p> </li> </ul> <p><em>Reasons to use redux/toolkit</em></p> <ul> <li>Easier to write and understand reducers with ImmerJs</li> <li>Slicers really concise</li> <li>The API encourages you to use a normalized state and provides you with practical CRUD function to update data in your store</li> </ul> <p>As redux/toolkit is the recommended library by the redux team, you should give it a try. Slicers + ImmerJS = 🔥 : your code is shorter and easier to understand. Moreover the createEntityAdapter will help you to normalize your state and write a new reducer faster thanks to the many CRUD functions provided.</p><![CDATA[Demystifying Building Native Modules for React Native]]>/2020/04/react-native-bridge-module//2020/04/react-native-bridge-module/Fri, 17 Apr 2020 00:00:00 GMT<style> .gatsby-highlight { font-size: 0.7rem; } a { font-size: 0.8rem; } code { font-family: "SFMono-Regular", Consolas, "Liberation Mono", Menlo, Courier, monospace; line-height: normal; background: rgba(135,131,120,0.15); color: #EB5757; border-radius: 3px; padding: 0.2em 0.4em; font-size: 0.7rem; } </style> <p>You’ve just released a new native SDK. Could you bring more business to your company by releasing on a cross-platform development framework?</p> <p>Maybe you’re a developer, longing for a native module of an open-source platform API (or even the third-party SDK above 🙄) to be released. Could you fast-track the development yourself?</p> <p>You could build <em>your own</em> module. With React Native, this becomes a straightforward process.</p> <p>Let’s talk about React Native for a second.</p> <h2>Why React Native</h2> <p>For those unfamiliar with React Native: it is a cross-platform development framework allowing you to create native apps for iOS and Android. Using React, you can maintain two platforms using the same codebase, allowing faster development, iteration and knowledge-sharing.</p> <p>With this framework, we have two sides; JavaScript and Native. Between the two is a bridge, allowing bidirectional and asynchronous communication. This is the power of React Native, on top of a multitude of other benefits.</p> <p>In 2020, we see the likes of Facebook, Bloomberg and Shopify <a href="https://reactnative.dev/showcase">[1]</a> using React Native to develop their mobile applications, amongst others in the Fortune 500. With over 11 million Javascript developers <a href="https://www.slashdata.co/free-resources/state-of-the-developer-nation-17th-edition?utm_source=BlogPost&#x26;utm_medium=Text">[2]</a> and more companies switching to React Native, releasing your technology for React Native could bring more growth.</p> <p>We’ve seen other cross-platform development frameworks like Flutter and Xamarin climb the ranks over the last few years, but React Native is still gaining popularity and ever-increasing its performance as it gets more mature.</p> <p>So hopefully that's convinced you to consider releasing a Native Modules for React Native if you're releasing Native SDKs. Let’s break down how you’d do that.</p> <h2>Native to React Native</h2> <p>As previously mentioned, communication from the native world to React Native is asynchronous. What this means is that any values must be sent through asynchronous callbacks, promises and events, each sent on a batched message queue. This is the architecture as of April 2020; there is currently a re-architecture of the React Native internals and due to be released mid-2020 <a href="https://github.com/react-native-community/discussions-and-proposals/issues/40">[3]</a>.</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 554px; " > <a class="gatsby-resp-image-link" href="/static/4d25817f9aa7c8bed5fb77401dc050db/04abd/rn-bridge-architecture-diagram.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 56.21621621621622%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAALCAYAAAB/Ca1DAAAACXBIWXMAAAsSAAALEgHS3X78AAABNUlEQVQoz32TW4+CQAyF5///MhLjgwlReIRELgIKylXA7n5NxrAEtkmRKT2np51q5Neez6e4ritN03CUz+ejbt+Xsb3z7XaTeZ7F8KiqStq2lbquZRiGTaKl/Xc2RVFImqbiOI4ST9O0mUjBJElUxVrh0g0ktAxxlmVSlqUCLDGKGQXfPc/Tol3X7ao3r9dLq/OBli04DEPp+15ja3u/3995Y7YAxZQwjmM5n886WBTRFnGUWwDk/OLjOKqA+/2uWIgQcblcxJDAbA6Hg0RRpEmQksQIUAIZqnDIiIFDwPV6VYzv+7opessoIZH27OwAoHJ5SUujiDXyMMgNAAipDiFEeZ7/mdHWbS7jiAKve0iAlo/Ho64P8mn38Xjs7uMW+feWYQ2CQE6nkw6YSus9XIP2/kH4D2VMWK6BzAlnAAAAAElFTkSuQmCC'); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="rn-bridge-architecture-diagram.png" title="rn-bridge-architecture-diagram.png" src="/static/4d25817f9aa7c8bed5fb77401dc050db/04abd/rn-bridge-architecture-diagram.png" srcset="/static/4d25817f9aa7c8bed5fb77401dc050db/1d79a/rn-bridge-architecture-diagram.png 185w, /static/4d25817f9aa7c8bed5fb77401dc050db/1efb2/rn-bridge-architecture-diagram.png 370w, /static/4d25817f9aa7c8bed5fb77401dc050db/04abd/rn-bridge-architecture-diagram.png 554w" sizes="(max-width: 554px) 100vw, 554px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <p>On the native side:</p> <ul> <li>The main thread is responsible for the UI.</li> <li>The shadow queue is responsible for layout calculations.</li> <li>Each native module has its own thread (Android shares a thread pool)</li> </ul> <p>On the JavaScript side:</p> <ul> <li>JavaScript VM thread which runs the bundled JS code and sends instructions to the native threads via the bridge.</li> </ul> <p>The two sides communicate over the bridge using the message queue.</p> <p>JavaScript knows about your native modules at runtime - there’s a JSON representation of each Native Module (consisting of module id, method id and arguments) and we can call methods on the Native Module this way.</p> <p>For the other direction, Native to JavaScript, we can use promises, callbacks and event to transfer data.</p> <p>To read more about the React Native internals and the upcoming re-architecture, read <a href="https://www.reactnative.guide/3-react-native-internals/3.1-react-native-internals.html">this article [4]</a>.</p> <h2>Module structure</h2> <p>To get started building your native module, there are a number of tools to help set up the skeleton of your project. For example:</p> <ul> <li><a href="https://github.com/react-native-community/bob">https://github.com/react-native-community/bob</a></li> <li><a href="https://github.com/brodybits/create-react-native-module">https://github.com/brodybits/create-react-native-module</a></li> <li><a href="https://github.com/peggyrayzis/react-native-create-bridge">https://github.com/peggyrayzis/react-native-create-bridge</a></li> </ul> <p>We like Bob, as your library will then come pre-configured with TypeScript and support for Kotlin and Swift. If you've got any other favourites, let us know!</p> <p>For project structure, if you’ve chosen Bob, the library should now have the following project structure:</p> <div class="gatsby-highlight" data-language="none"><pre class="language-none"><code class="language-none">- /ios - /android - /src - index.tsx - /yourExampleApp - package.json - ... - ...</code></pre></div> <p>In the iOS and Android folders, you’ll want to have your native SDK code - either available in your project or using git submodules to keep the version control.</p> <h2>Exposing iOS Modules</h2> <p>On the iOS Side, there are two languages at play: Objective-C and Swift. You can write the native module entirely in Objective-C, or write methods in Swift and expose them to Objective-C by using the <code>@objc</code> attribute.</p> <p>The steps are then as follows:</p> <ol> <li> <p>Create a Bridging Header file (Objective-C, <code>.h</code> file) - Here you'll import:</p> <div class="gatsby-highlight" data-language="objectivec"><pre class="language-objectivec"><code class="language-objectivec"><span class="token macro property"><span class="token directive-hash">#</span><span class="token directive keyword">import</span> <span class="token string">"React/RCTBridgeModule.h"</span></span> <span class="token macro property"><span class="token directive-hash">#</span><span class="token directive keyword">import</span> <span class="token expression"><span class="token operator">&lt;</span>React<span class="token operator">/</span>RCTLog<span class="token punctuation">.</span>h<span class="token operator">></span></span></span> <span class="token macro property"><span class="token directive-hash">#</span><span class="token directive keyword">import</span> <span class="token expression"><span class="token operator">&lt;</span>React<span class="token operator">/</span>RCTEventEmitter<span class="token punctuation">.</span>h<span class="token operator">></span></span></span> <span class="token macro property"><span class="token directive-hash">#</span><span class="token directive keyword">import</span> <span class="token expression"><span class="token operator">&lt;</span>React<span class="token operator">/</span>RCTConvert<span class="token punctuation">.</span>h<span class="token operator">></span></span></span></code></pre></div> </li> <li> <p>Create a Bridging Command file (Objective-C, <code>.m</code> file), expose modules with <code>RCT_EXPORT_MODULE</code> and Expose methods with <code>RCT_EXPORT_METHOD</code></p> <div class="gatsby-highlight" data-language="objectivec"><pre class="language-objectivec"><code class="language-objectivec"><span class="token macro property"><span class="token directive-hash">#</span><span class="token directive keyword">import</span> <span class="token expression"><span class="token operator">&lt;</span>React<span class="token operator">/</span>RCTBridgeModule<span class="token punctuation">.</span>h<span class="token operator">></span></span></span> <span class="token macro property"><span class="token directive-hash">#</span><span class="token directive keyword">import</span> <span class="token expression"><span class="token operator">&lt;</span>React<span class="token operator">/</span>RCTEventEmitter<span class="token punctuation">.</span>h<span class="token operator">></span></span></span> <span class="token macro property"><span class="token directive-hash">#</span><span class="token directive keyword">import</span> <span class="token string">"MyModule.h"</span></span> <span class="token keyword">@interface</span> <span class="token function">RCT_EXTERN_MODULE</span><span class="token punctuation">(</span>MyModule<span class="token punctuation">,</span> NSObject<span class="token punctuation">)</span> <span class="token function">RCT_EXTERN_METHOD</span><span class="token punctuation">(</span>myMethodWithNoParams<span class="token punctuation">)</span> <span class="token function">RCT_EXTERN_METHOD</span><span class="token punctuation">(</span>myMethodWithAPromise<span class="token punctuation">:</span><span class="token punctuation">(</span>NSString<span class="token punctuation">)</span>input resolver<span class="token punctuation">:</span><span class="token punctuation">(</span>RCTPromiseResolveBlock<span class="token punctuation">)</span>resolve rejecter<span class="token punctuation">:</span><span class="token punctuation">(</span>RCTPromiseRejectBlock<span class="token punctuation">)</span>reject<span class="token punctuation">)</span> <span class="token operator">+</span> <span class="token punctuation">(</span>BOOL<span class="token punctuation">)</span>requiresMainQueueSetup <span class="token punctuation">{</span> <span class="token keyword">return</span> YES<span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token keyword">@end</span></code></pre></div> </li> <li> <p>Define event emitters, implementing the Delegate pattern.</p> <div class="gatsby-highlight" data-language="objectivec"><pre class="language-objectivec"><code class="language-objectivec"><span class="token macro property"><span class="token directive-hash">#</span><span class="token directive keyword">import</span> <span class="token expression"><span class="token operator">&lt;</span>React<span class="token operator">/</span>RCTBridgeModule<span class="token punctuation">.</span>h<span class="token operator">></span></span></span> <span class="token macro property"><span class="token directive-hash">#</span><span class="token directive keyword">import</span> <span class="token expression"><span class="token operator">&lt;</span>React<span class="token operator">/</span>RCTEventEmitter<span class="token punctuation">.</span>h<span class="token operator">></span></span></span> <span class="token macro property"><span class="token directive-hash">#</span><span class="token directive keyword">import</span> <span class="token expression"><span class="token operator">&lt;</span>React<span class="token operator">/</span>RCTConvert<span class="token punctuation">.</span>h<span class="token operator">></span></span></span> <span class="token keyword">@interface</span> <span class="token function">RCT_EXTERN_MODULE</span><span class="token punctuation">(</span>MyModuleEmitter<span class="token punctuation">,</span> RCTEventEmitter<span class="token punctuation">)</span> <span class="token comment">// Create a singleton for the EventEmitter class</span> <span class="token operator">-</span> <span class="token punctuation">(</span>id<span class="token punctuation">)</span>allocWithZone<span class="token punctuation">:</span><span class="token punctuation">(</span>NSZone <span class="token operator">*</span><span class="token punctuation">)</span>zone <span class="token punctuation">{</span> <span class="token keyword">static</span> MyModuleEventEmitter <span class="token operator">*</span>shared <span class="token operator">=</span> nil<span class="token punctuation">;</span> <span class="token keyword">static</span> dispatch_once_t onceToken<span class="token punctuation">;</span> <span class="token function">dispatch_once</span><span class="token punctuation">(</span><span class="token operator">&amp;</span>onceToken<span class="token punctuation">,</span> <span class="token operator">^</span><span class="token punctuation">{</span> shared <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token keyword">super</span> allocWithZone<span class="token punctuation">:</span>zone<span class="token punctuation">]</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">return</span> shared<span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token operator">-</span> <span class="token punctuation">(</span>BOOL<span class="token punctuation">)</span>requiresMainQueueSetup <span class="token punctuation">{</span> <span class="token keyword">return</span> YES<span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token keyword">@end</span></code></pre></div> </li> </ol> <p>For Swift, a good pattern to use is the Observer-Command-Emitter pattern:</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 651px; " > <a class="gatsby-resp-image-link" href="/static/8bfa2829ead9f47dd957bd40d1a1ac04/1ac66/observer-command-emitter-diagram.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 43.24324324324324%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAJCAYAAAAywQxIAAAACXBIWXMAAAsSAAALEgHS3X78AAABlUlEQVQoz32Sb2+aUBjF+f6fY9lru73p0iWdscUUu8Ta6AQJitCRooLC+CMX7m9XNI1uTZ/k5HlygXOfcw4aNJxXFEXouk6RFXxYUiLbfhjl27FmWgLvJWO9XhEEgepr4igm2AVMV1PsyCaMQ5wk5Gnr8hS7TNLf7Jv6xCsvoPX78Dze4boOs5mNZVlYUwvDM7iyr7j2rhnaQ+4Dk44/oOMafHsdke7fV6ANBjB3xcVhkiTtbU3TUNfHTXIpSKj4g2ArCnbqnTzPyURJoZ4VsmpnrdcrceaV+qRuCQ7YbDb/2CVbr95wqlzsuVHbflGbf/UeuU1MtG43PxEeyI4+RJuIRh7JD/3c9LORTMn+GTvcr0x0hWHhoxmGkry4lLxLdv95U4mKqhYIFUaxL0nTlLIoaUSNVBfLWtlTCbRH5eF4kqqEfSU1alNeOIs24Z7bw3g18FZLxukLP8IJXYWHxCHOErh04BhKtx/z/CtU6U4xTVMlPWO5WGJGJnfBHcbGwI98budDPo1u+Dz6Tmemsy3Sd3+bv4euqv2jNNmJAAAAAElFTkSuQmCC'); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="observer-command-emitter-diagram.png" title="observer-command-emitter-diagram.png" src="/static/8bfa2829ead9f47dd957bd40d1a1ac04/1ac66/observer-command-emitter-diagram.png" srcset="/static/8bfa2829ead9f47dd957bd40d1a1ac04/1d79a/observer-command-emitter-diagram.png 185w, /static/8bfa2829ead9f47dd957bd40d1a1ac04/1efb2/observer-command-emitter-diagram.png 370w, /static/8bfa2829ead9f47dd957bd40d1a1ac04/1ac66/observer-command-emitter-diagram.png 651w" sizes="(max-width: 651px) 100vw, 651px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <p>This ties together your observer which implements the delegate, the event emitter which triggers events and the commands which will be the methods to expose to JavaScript. In the above code snippets, you've seen the Objective-C <code>.m</code> files for the above.</p> <h2>Exposing Android Modules</h2> <p>The steps for Android are similar:</p> <ol> <li> <p>Create a <code>ReactPackage</code> to declare the modules to expose:</p> <div class="gatsby-highlight" data-language="kotlin"><pre class="language-kotlin"><code class="language-kotlin"><span class="token keyword">package</span> com<span class="token punctuation">.</span>mymodule <span class="token keyword">import</span> java<span class="token punctuation">.</span>util<span class="token punctuation">.</span>Arrays <span class="token keyword">import</span> com<span class="token punctuation">.</span>facebook<span class="token punctuation">.</span>react<span class="token punctuation">.</span>ReactPackage <span class="token keyword">import</span> com<span class="token punctuation">.</span>facebook<span class="token punctuation">.</span>react<span class="token punctuation">.</span>bridge<span class="token punctuation">.</span>NativeModule <span class="token keyword">import</span> com<span class="token punctuation">.</span>facebook<span class="token punctuation">.</span>react<span class="token punctuation">.</span>bridge<span class="token punctuation">.</span>ReactApplicationContext <span class="token keyword">import</span> com<span class="token punctuation">.</span>facebook<span class="token punctuation">.</span>react<span class="token punctuation">.</span>uimanager<span class="token punctuation">.</span>ViewManager <span class="token keyword">class</span> MyPackage <span class="token operator">:</span> ReactPackage <span class="token punctuation">{</span> <span class="token keyword">override</span> <span class="token keyword">fun</span> <span class="token function">createNativeModules</span><span class="token punctuation">(</span>reactContext<span class="token operator">:</span> ReactApplicationContext<span class="token punctuation">)</span><span class="token operator">:</span> List<span class="token operator">&lt;</span>NativeModule<span class="token operator">></span> <span class="token punctuation">{</span> <span class="token keyword">return</span> Arrays<span class="token punctuation">.</span>asList<span class="token operator">&lt;</span>NativeModule<span class="token operator">></span><span class="token punctuation">(</span><span class="token function">MyModule</span><span class="token punctuation">(</span>reactContext<span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">}</span> <span class="token keyword">override</span> <span class="token keyword">fun</span> <span class="token function">createViewManagers</span><span class="token punctuation">(</span>reactContext<span class="token operator">:</span> ReactApplicationContext<span class="token punctuation">)</span><span class="token operator">:</span> List<span class="token operator">&lt;</span>ViewManager<span class="token operator">&lt;</span><span class="token operator">*</span><span class="token punctuation">,</span> <span class="token operator">*</span><span class="token operator">></span><span class="token operator">></span> <span class="token punctuation">{</span> <span class="token keyword">return</span> emptyList<span class="token operator">&lt;</span>ViewManager<span class="token operator">&lt;</span><span class="token operator">*</span><span class="token punctuation">,</span> <span class="token operator">*</span><span class="token operator">></span><span class="token operator">></span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span></code></pre></div> </li> <li> <p>Create your module extending <code>ReactContextBaseJavaModule</code> to declare the methods to expose using <code>@ReactMethod</code> . Implement listeners to emit events.</p> <div class="gatsby-highlight" data-language="kotlin"><pre class="language-kotlin"><code class="language-kotlin"><span class="token keyword">package</span> com<span class="token punctuation">.</span>mypackage <span class="token keyword">import</span> com<span class="token punctuation">.</span>facebook<span class="token punctuation">.</span>react<span class="token punctuation">.</span>bridge<span class="token punctuation">.</span>_ <span class="token keyword">import</span> com<span class="token punctuation">.</span>facebook<span class="token punctuation">.</span>react<span class="token punctuation">.</span>modules<span class="token punctuation">.</span>core<span class="token punctuation">.</span>DeviceEventManagerModule<span class="token punctuation">.</span>RCTDeviceEventEmitter <span class="token keyword">import</span> java<span class="token punctuation">.</span>util<span class="token punctuation">.</span>\<span class="token operator">*</span> <span class="token keyword">enum</span> <span class="token keyword">class</span> <span class="token function">MyModuleError</span><span class="token punctuation">(</span><span class="token keyword">val</span> errorCode<span class="token operator">:</span> String<span class="token punctuation">,</span> <span class="token keyword">val</span> message<span class="token operator">:</span> String<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token function">INPUT_NOT_RECOGNISED</span><span class="token punctuation">(</span> <span class="token string">"INPUT_NOT_RECOGNISED"</span><span class="token punctuation">,</span> <span class="token string">"Input was not of the correct format."</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">}</span> <span class="token keyword">class</span> <span class="token function">MyModule</span><span class="token punctuation">(</span>reactContext<span class="token operator">:</span> ReactApplicationContext<span class="token punctuation">)</span> <span class="token operator">:</span> <span class="token function">ReactContextBaseJavaModule</span><span class="token punctuation">(</span>reactContext<span class="token punctuation">)</span><span class="token punctuation">,</span> MyListener <span class="token punctuation">{</span> <span class="token keyword">companion</span> <span class="token keyword">object</span> <span class="token punctuation">{</span> <span class="token keyword">const</span> <span class="token keyword">val</span> MODULE_NAME <span class="token operator">=</span> <span class="token string">"MyModule"</span> <span class="token punctuation">}</span> <span class="token keyword">override</span> <span class="token keyword">fun</span> <span class="token function">getName</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token operator">:</span> String <span class="token punctuation">{</span> <span class="token keyword">return</span> MODULE_NAME <span class="token punctuation">}</span> <span class="token keyword">private</span> <span class="token keyword">fun</span> <span class="token function">createErrorHandler</span><span class="token punctuation">(</span>promise<span class="token operator">:</span> Promise<span class="token punctuation">)</span><span class="token operator">:</span> <span class="token punctuation">(</span>TokenError<span class="token punctuation">)</span> <span class="token operator">-></span> Unit <span class="token punctuation">{</span> <span class="token keyword">return</span> <span class="token keyword">fun</span><span class="token punctuation">(</span>error<span class="token operator">:</span> TokenError<span class="token punctuation">)</span> <span class="token punctuation">{</span> promise<span class="token punctuation">.</span><span class="token function">reject</span><span class="token punctuation">(</span>error<span class="token punctuation">.</span>errorCode<span class="token punctuation">,</span> error<span class="token punctuation">.</span>message<span class="token punctuation">)</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> <span class="token annotation builtin">@ReactMethod</span> <span class="token keyword">fun</span> <span class="token function">myMethodWithNoParams</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token function">doSomething</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token annotation builtin">@ReactMethod</span> <span class="token keyword">fun</span> <span class="token function">myMethodWithAPromise</span><span class="token punctuation">(</span>input<span class="token operator">:</span> String<span class="token punctuation">,</span> promise<span class="token operator">:</span> Promise<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">val</span> handleError <span class="token operator">=</span> <span class="token function">createErrorHandler</span><span class="token punctuation">(</span>promise<span class="token punctuation">)</span> <span class="token function">doSomethingWithInputSucceeds</span><span class="token punctuation">(</span>input<span class="token punctuation">)</span><span class="token operator">?</span><span class="token punctuation">.</span> let <span class="token punctuation">{</span> promise<span class="token punctuation">.</span><span class="token function">resolve</span><span class="token punctuation">(</span><span class="token string">"MY_METHOD_SUCCESS"</span><span class="token punctuation">)</span> <span class="token punctuation">}</span> <span class="token operator">?:</span> kotlin<span class="token punctuation">.</span><span class="token function">run</span> <span class="token punctuation">{</span> <span class="token function">handleError</span><span class="token punctuation">(</span>ModuleError<span class="token punctuation">.</span>INPUT_NOT_RECOGNISED<span class="token punctuation">)</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> <span class="token keyword">override</span> <span class="token keyword">fun</span> <span class="token function">onEventSuccess</span><span class="token punctuation">(</span>result<span class="token operator">:</span> String<span class="token punctuation">,</span> message<span class="token operator">:</span> String<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">val</span> body<span class="token operator">:</span> WritableMap <span class="token operator">=</span> Arguments<span class="token punctuation">.</span><span class="token function">makeNativeMap</span><span class="token punctuation">(</span><span class="token function">mapOf</span><span class="token punctuation">(</span> <span class="token string">"result"</span> <span class="token keyword">to</span> result<span class="token punctuation">,</span> <span class="token string">"message"</span> <span class="token keyword">to</span> message <span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token keyword">this</span><span class="token punctuation">.</span>reactApplicationContext<span class="token punctuation">.</span><span class="token function">getJSModule</span><span class="token punctuation">(</span>RCTDeviceEventEmitter<span class="token operator">::</span><span class="token keyword">class</span><span class="token punctuation">.</span>java<span class="token punctuation">)</span> <span class="token punctuation">.</span><span class="token function">emit</span><span class="token punctuation">(</span><span class="token string">"MyModule/eventSucceeded"</span><span class="token punctuation">,</span> body<span class="token punctuation">)</span> <span class="token punctuation">}</span> <span class="token keyword">override</span> <span class="token keyword">fun</span> <span class="token function">onEventFailure</span><span class="token punctuation">(</span>error<span class="token operator">:</span> String<span class="token punctuation">,</span> message<span class="token operator">:</span> String<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">val</span> body<span class="token operator">:</span> WritableMap <span class="token operator">=</span> Arguments<span class="token punctuation">.</span><span class="token function">makeNativeMap</span><span class="token punctuation">(</span><span class="token function">mapOf</span><span class="token punctuation">(</span> <span class="token string">"error"</span> <span class="token keyword">to</span> error<span class="token punctuation">,</span> <span class="token string">"message"</span> <span class="token keyword">to</span> message <span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token keyword">this</span><span class="token punctuation">.</span>reactApplicationContext<span class="token punctuation">.</span><span class="token function">getJSModule</span><span class="token punctuation">(</span>RCTDeviceEventEmitter<span class="token operator">::</span><span class="token keyword">class</span><span class="token punctuation">.</span>java<span class="token punctuation">)</span> <span class="token punctuation">.</span><span class="token function">emit</span><span class="token punctuation">(</span><span class="token string">"MyModule/eventFailed"</span><span class="token punctuation">,</span> body<span class="token punctuation">)</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span></code></pre></div> </li> </ol> <h2>Your new module</h2> <p>Now you’ve exposed your native modules and methods to React Native and can continue to build on top of them. What’s left is to import them from <code>NativeModules</code> from <code>react-native</code> and you’re ready to create your app:</p> <p>In <code>/src</code> of your Native Module:</p> <div class="gatsby-highlight" data-language="javascript"><pre class="language-javascript"><code class="language-javascript"><span class="token keyword">import</span> <span class="token punctuation">{</span> NativeModules<span class="token punctuation">,</span> DeviceEventEmitter<span class="token punctuation">,</span> NativeEventEmitter<span class="token punctuation">,</span> <span class="token punctuation">}</span> <span class="token keyword">from</span> <span class="token string">"react-native"</span><span class="token punctuation">;</span> <span class="token keyword">export</span> <span class="token keyword">const</span> <span class="token punctuation">{</span> MyModule<span class="token punctuation">,</span> MyModuleEventEmitter <span class="token punctuation">}</span> <span class="token operator">=</span> NativeModules<span class="token punctuation">;</span></code></pre></div> <p>Now you're ready to go. Just <code>yarn add react-native-myModule</code> and import the above <code>MyModule</code> and emitters to get going.</p> <p>To launch your development experience and quality to the next level:</p> <ol> <li><em>Add a React Native deployment pipeline</em> - <a href="https://blog.theodo.com/2019/04/react-native-deployment-pipeline/">https://blog.theodo.com/2019/04/react-native-deployment-pipeline/</a></li> <li><em>Add end-to-end tests with Detox -</em> <a href="https://github.com/wix/Detox">https://github.com/wix/Detox</a></li> <li><em>Add support for offline</em> - <a href="https://dev.to/reactnativeradio/rnr-157-building-great-offline-ready-apps-in-react-native-with-josh-warwick">https://dev.to/reactnativeradio/rnr-157-building-great-offline-ready-apps-in-react-native-with-josh-warwick</a></li> </ol> <p>What a great development set up!</p> <h2>Summary</h2> <p>Once that’s all ready, your module is tested, you’re ready to publish. Very straightforward with <code>npm publish</code><a href="https://docs.npmjs.com/cli/publish">[5]</a>.</p> <p>There you have it. A native module which can now be released to the JavaScript tech community - perhaps bringing in more customers for your business or fast-tracking your next development track.</p> <p>Hopefully this article gave you an overall idea of where to start building your bridge module. If you have any questions, <a href="https://www.theodo.co.uk/contact">let us know</a>, we'd be happy to help!</p> <p><em>For more resources, the following are some good resources:</em></p> <ul> <li><a href="https://reactnative.dev/docs/native-modules-ios">https://reactnative.dev/docs/native-modules-ios</a></li> <li><a href="https://reactnative.dev/docs/native-modules-android">https://reactnative.dev/docs/native-modules-android</a></li> <li><a href="https://hackernoon.com/react-native-bridge-for-ios-and-android-43feb9712fcb">https://hackernoon.com/react-native-bridge-for-ios-and-android-43feb9712fcb</a></li> <li><a href="http://nightlyclosures.com/2018/02/09/writing-a-react-native-ios-module-in-swift/">http://nightlyclosures.com/2018/02/09/writing-a-react-native-ios-module-in-swift/</a></li> </ul> <p><em>Sources + links:</em></p> <ol> <li><a href="https://reactnative.dev/showcase">https://reactnative.dev/showcase</a></li> <li><a href="https://www.slashdata.co/free-resources/state-of-the-developer-nation-17th-edition?utm_source=BlogPost&#x26;utm_medium=Text">https://www.slashdata.co/free-resources/state-of-the-developer-nation-17th-edition?utm<em>source=BlogPost&#x26;utm</em>medium=Text</a></li> <li><a href="https://github.com/react-native-community/discussions-and-proposals/issues/40">https://github.com/react-native-community/discussions-and-proposals/issues/40</a></li> <li><a href="https://www.reactnative.guide/3-react-native-internals/3.1-react-native-internals.html">https://www.reactnative.guide/3-react-native-internals/3.1-react-native-internals.html</a></li> <li><a href="https://docs.npmjs.com/cli/publish">https://docs.npmjs.com/cli/publish</a></li> </ol><![CDATA[How to convert an imperative library to a declarative React component: the Sketchfab Viewer]]>/2020/04/sketchfab-react-part-1//2020/04/sketchfab-react-part-1/Fri, 10 Apr 2020 00:00:00 GMT<p>I love using React for multiple reasons. I like the writing style, I like the tooling, I like the ecosystem.</p> <p>But sometimes you run into a library that doesn't have a React adapter. It can either be pretty old, kind of niche, closed source, etc.</p> <p>In this series of articles, we’ll look into how to convert an existing imperative JS library into a declarative React component.</p> <p>We’ll use the <a href="https://sketchfab.com/3d-viewer">Sketchfab Viewer</a> as an example. Sketchfab develops an amazing 3d suite that includes a very configurable web viewer for use with their models.</p> <p>You can see that <a href="https://sketchfab.com/developers/viewer">their doc</a> is pretty extensive, if a bit messy. It includes a lot of examples and it took me a long time to run into a use case that was not already covered. Yet you can also see that they use an imperative style of JS that doesn’t fit well into the React patterns.</p> <div class="gatsby-highlight" data-language="js"><pre class="language-js"><code class="language-js">api<span class="token punctuation">.</span><span class="token function">addEventListener</span><span class="token punctuation">(</span><span class="token string">"viewerready"</span><span class="token punctuation">,</span> <span class="token keyword">function</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span><span class="token string">"Viewer is ready"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre></div> <p>We are going to build a React function component on top of that library. It will accept a config object to declaratively change a color on the model. It will also expose a callback to react to a click event on a part of the model. And we'll use hooks because they're fun!</p> <p>You can find the final code for this part <a href="https://github.com/aldebout/sketchfab-react/tree/part1">here</a>.</p> <ul> <li>Part 1: Initializing and exposing the api object</li> <li>Part 2: Working on it!</li> </ul> <h1>Initializing and exposing the api object</h1> <p>Throughout this project we'll work with this fancy chair, courtesy of the <a href="https://labs.sketchfab.com/experiments/configurator/">official Sketchfab examples</a>.</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/7a23d7327a4eb4d201903cbbedb79637/50383/chair-demo.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 103.78378378378379%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAVCAYAAABG1c6oAAAACXBIWXMAAAsSAAALEgHS3X78AAADlklEQVQ4y4WU229UVRTGz99jCFEaB8tcz2XOZebcz5lzOtPC4BTaWluF1kKtwoRKxtAYEALRqFF8ECKYRpK+aDTxEjC8mJgY4wOakCjBJzEalAf9XHt39niGNvHhS87lW7+9195rLalcLqNSqXDJsjyQoihbpKoaVE2Doqr8XXhFPGNJDwNUMg9EwVpGcrGASmEP5FJxyCdiGUfKQrLB1Wr1P/Xfo5nDaB45gejALH/P+gVDEqAhAEnX9U0ZBjS5gubqWcx/8j2m1m9yNZeOQS2XyGMMwaVtIX0ZhgldVVBLJ3Bw/Svsf+9TjL22Dv/0ZbRWz0DXFL4g8wqG9DDIIIOQaVnQ5TLi51/GJAHTCx8gfvUS9BfPIVw+CVOvDoACKm0H4jDT3ATSv/Yb6+hc/RLx2Uvw195B6dBJuItdWOZwDPNKW3bVB1m2DbWYhzc5h0Of/4DO+58hOX8F/qm3sWf2OOzDXdQs8/+AmwaN1WS+gKA1jtWvb6F3+2/Mf/Qd9r77MU85//QJ+Mu97YGDXfGdkQg6sbCA7sYGLv56D5cBvPnnP3jl9gO8cPMOpq7cQHz+Gvb1zsGsqjwbEc9YUq1WA1O9Xucr2n6E0z/dw1UCXSS99QB4/T5w4TfgzF2g++1fWLx2HY32k3QpOsXZEAwmyXEcCLmuC5vAydxzWPjwC7z0zc849eMfWLv1Oz3/gqWNGzi61kPSCGlxC67nUcxmnGBIvu9DKAgChCSHoKltYG6vB2viIOanxjG7L4CsaDjglvHMhMszYd6gL8GQwjBEFEVccRyjEUfw4hSLbQczqQVVr2HazaM7HcILYwSeg+OTNuJGg/uFBEPiEPrJlCQJV0o61tbRiGixZAxzrRp68ymaYym8KMGRto2n2g2EtHDajxGMAVDAwkaK6ZaHpXYdbhDzdFLPxEqHdhUGcCnV/amP5Y7HvVuAIlUuena8AAutKmLHQEVWacaVkHsij6Vxan65RCNKQZUGwrMtkzKgM8/E85SDzME6rgffsdDxZRQKReRyOeza9Rge2bETM0EBo7kRPJ7bjVKpBMvQ4To2PM9HSDtnd8EkeXT1Qja1G6tFTVORLxQ4cGRkBDt2Pop2fZS6J4fc7lGazmU+CFjdsZLJVspQHTKgRSY249icZCO9WCxilNqw6ahwDAXlisxbzKI6ZH5Rv6IWJdElWbFvVr+leG/TjldWVujQEw4TPgbMin37F7VpThn3aBMMAAAAAElFTkSuQmCC'); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="The default Sketchfab chair" title="The default Sketchfab chair" src="/static/7a23d7327a4eb4d201903cbbedb79637/50383/chair-demo.png" srcset="/static/7a23d7327a4eb4d201903cbbedb79637/1d79a/chair-demo.png 185w, /static/7a23d7327a4eb4d201903cbbedb79637/1efb2/chair-demo.png 370w, /static/7a23d7327a4eb4d201903cbbedb79637/50383/chair-demo.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <h2>Start the project with create-react-app</h2> <p>First things first, let's generate a <a href="https://github.com/facebook/create-react-app">basic CRA project</a>:</p> <div class="gatsby-highlight" data-language="shell"><pre class="language-shell"><code class="language-shell">npx create-react-app sketchfab-react</code></pre></div> <p>In <code>src/</code>, add a <code>Viewer.jsx</code> file and create an empty component inside.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">import</span> React <span class="token keyword">from</span> <span class="token string">"react"</span><span class="token punctuation">;</span> <span class="token keyword">export</span> <span class="token keyword">const</span> <span class="token function-variable function">Viewer</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span></span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span></span><span class="token punctuation">></span></span><span class="token punctuation">;</span></code></pre></div> <p>Add that component inside App.jsx</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">import</span> React <span class="token keyword">from</span> <span class="token string">"react"</span><span class="token punctuation">;</span> <span class="token keyword">import</span> <span class="token string">"./App.css"</span><span class="token punctuation">;</span> <span class="token keyword">import</span> <span class="token punctuation">{</span> Viewer <span class="token punctuation">}</span> <span class="token keyword">from</span> <span class="token string">"./Viewer"</span><span class="token punctuation">;</span> <span class="token keyword">function</span> <span class="token function">App</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">return</span> <span class="token punctuation">(</span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span> <span class="token attr-name">className</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>App<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">Viewer</span></span> <span class="token punctuation">/></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>div</span><span class="token punctuation">></span></span> <span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token keyword">export</span> <span class="token keyword">default</span> App<span class="token punctuation">;</span></code></pre></div> <p>Now let's import the library. Sketchfab provides a <a href="https://www.npmjs.com/package/@sketchfab/viewer-api">package install through npm</a> but that's not always the case for other libraries, so we'll include the source in the head of <code>public/index.html</code>.</p> <div class="gatsby-highlight" data-language="html"><pre class="language-html"><code class="language-html"><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>script</span> <span class="token attr-name">type</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>text/javascript<span class="token punctuation">"</span></span> <span class="token attr-name">src</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>https://static.sketchfab.com/api/sketchfab-viewer-1.7.1.js<span class="token punctuation">"</span></span> <span class="token punctuation">></span></span><span class="token script"></span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>script</span><span class="token punctuation">></span></span></code></pre></div> <p>This import will make <code>window.Sketchfab</code> available globally. If you're using Typescript, you will need to ignore it.</p> <h2>Render the Sketchfab Viewer in an iframe</h2> <p>According to the docs, we need to create an iframe and give it to the Sketchfab constructor to render the Viewer.</p> <p>The way to do this in React is, in Viewer.jsx:</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">import</span> React<span class="token punctuation">,</span> <span class="token punctuation">{</span> useEffect<span class="token punctuation">,</span> useRef <span class="token punctuation">}</span> <span class="token keyword">from</span> <span class="token string">"react"</span><span class="token punctuation">;</span> <span class="token comment">// Our wonderful chair model</span> <span class="token keyword">const</span> <span class="token constant">MODEL_UID</span> <span class="token operator">=</span> <span class="token string">"c632823b6c204797bd9b95dbd9f53a06"</span><span class="token punctuation">;</span> <span class="token keyword">export</span> <span class="token keyword">const</span> <span class="token function-variable function">Viewer</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token comment">// This ref will contain the actual iframe object</span> <span class="token keyword">const</span> viewerIframeRef <span class="token operator">=</span> <span class="token function">useRef</span><span class="token punctuation">(</span><span class="token keyword">null</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">const</span> ViewerIframe <span class="token operator">=</span> <span class="token punctuation">(</span> <span class="token operator">&lt;</span>iframe <span class="token comment">// We feed the ref to the iframe component to get the underlying DOM object</span> ref<span class="token operator">=</span><span class="token punctuation">{</span>viewerIframeRef<span class="token punctuation">}</span> title<span class="token operator">=</span><span class="token string">"sketchfab-viewer"</span> style<span class="token operator">=</span><span class="token punctuation">{</span><span class="token punctuation">{</span> height<span class="token operator">:</span> <span class="token number">400</span><span class="token punctuation">,</span> width<span class="token operator">:</span> <span class="token number">600</span> <span class="token punctuation">}</span><span class="token punctuation">}</span> <span class="token operator">/</span><span class="token operator">></span> <span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">useEffect</span><span class="token punctuation">(</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token comment">// Initialize the viewer</span> <span class="token keyword">let</span> client <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">window<span class="token punctuation">.</span>Sketchfab</span><span class="token punctuation">(</span>viewerIframeRef<span class="token punctuation">.</span>current<span class="token punctuation">)</span><span class="token punctuation">;</span> client<span class="token punctuation">.</span><span class="token function">init</span><span class="token punctuation">(</span><span class="token constant">MODEL_UID</span><span class="token punctuation">,</span> <span class="token punctuation">{</span> <span class="token function-variable function">success</span><span class="token operator">:</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token function-variable function">error</span><span class="token operator">:</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span><span class="token string">"Viewer error"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token comment">// We only want to initialize the viewer on first load, so we don't add any dependencies to useEffect</span> <span class="token punctuation">[</span><span class="token punctuation">]</span> <span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">return</span> ViewerIframe<span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">;</span></code></pre></div> <p>And you should have the viewer displaying!</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/54cd47ac648b8fd27441256b9b890893/50383/embed-on-page.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 50.810810810810814%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAKCAYAAAC0VX7mAAAACXBIWXMAAAsSAAALEgHS3X78AAABB0lEQVQoz62Q3UqEUBSFfSEpFUZpRI4eUzwRxdDY+CIGwgxdTK+Uc6GJFz5Ad9ET+A7+rI6GITGGQgsWbDZrf+y9BYzUti3quu7rMAwhiiJM0wThtikFIQSGYUCSJKRp2ueG/CBhChhFERRFgeu6uLYpLq9sEOrw2oaqqsiybBlwvz9A0zQwxsBuNzCcOxgmxQ3zoOs68jxfBnw5HuE4DnZBgN12g8d7hsB/wNb34XkeiqJYBnw6POOC/8rmJ661FdYrmf+TwLIsyLI8/4edO71/fOL1dOoHk/SNO/2ukwRxHKMsy5+ZSeAYPEfncpMbNk2Dqqr6k3676w+5WRuO4X/5nAT8s74AZm284RAJMxMAAAAASUVORK5CYII='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="The Sketchfab viewer embedded on a page" title="The Sketchfab viewer embedded on a page" src="/static/54cd47ac648b8fd27441256b9b890893/50383/embed-on-page.png" srcset="/static/54cd47ac648b8fd27441256b9b890893/1d79a/embed-on-page.png 185w, /static/54cd47ac648b8fd27441256b9b890893/1efb2/embed-on-page.png 370w, /static/54cd47ac648b8fd27441256b9b890893/50383/embed-on-page.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <h2>Expose the api object to access it imperatively</h2> <p>One of the great things about the Viewer API is that it allows us to transform our model from the browser. Here, we'll try to make the chair red.</p> <p>First we need to extract the api object and store it. We could put it in the state of our Viewer component, but we'll want to use it higher up in the tree, so we will <a href="https://reactjs.org/docs/hooks-reference.html#useref">pass a ref</a> as prop and give it the api.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">export</span> <span class="token keyword">const</span> <span class="token function-variable function">Viewer</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token parameter"><span class="token punctuation">{</span> apiRef <span class="token punctuation">}</span></span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token comment">// ...</span> <span class="token function-variable function">success</span><span class="token operator">:</span> <span class="token parameter">api</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token comment">// Store the initialized api inside the provided ref</span> apiRef<span class="token punctuation">.</span>current <span class="token operator">=</span> api<span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token comment">// ...</span> <span class="token punctuation">}</span><span class="token punctuation">;</span></code></pre></div> <p>Now, let's get it inside App.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">function</span> <span class="token function">App</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">const</span> apiRef <span class="token operator">=</span> <span class="token function">useRef</span><span class="token punctuation">(</span><span class="token keyword">null</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">return</span> <span class="token punctuation">(</span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span> <span class="token attr-name">className</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>App<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">Viewer</span></span> <span class="token attr-name">apiRef</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>apiRef<span class="token punctuation">}</span></span> <span class="token punctuation">/></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>div</span><span class="token punctuation">></span></span> <span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span></code></pre></div> <p>To check that it works, let's try to change the color of the background. <a href="https://sketchfab.com/developers/viewer/functions#api-background">There's a method for that!</a></p> <p>Let's add this code to <code>App.jsx</code>:</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">function</span> <span class="token function">App</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token comment">// ...</span> <span class="token keyword">const</span> <span class="token function-variable function">changeBackgroundColor</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> apiRef<span class="token punctuation">.</span>current<span class="token punctuation">.</span><span class="token function">setBackground</span><span class="token punctuation">(</span><span class="token punctuation">{</span> color<span class="token operator">:</span> <span class="token punctuation">[</span>Math<span class="token punctuation">.</span><span class="token function">random</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> Math<span class="token punctuation">.</span><span class="token function">random</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> Math<span class="token punctuation">.</span><span class="token function">random</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">;</span> <span class="token keyword">return</span> <span class="token punctuation">(</span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span> <span class="token attr-name">className</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>App<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>button</span> <span class="token attr-name">onClick</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>changeBackgroundColor<span class="token punctuation">}</span></span><span class="token punctuation">></span></span><span class="token plain-text">Change background color</span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>button</span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">Viewer</span></span> <span class="token attr-name">apiRef</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>apiRef<span class="token punctuation">}</span></span> <span class="token punctuation">/></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>div</span><span class="token punctuation">></span></span> <span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span></code></pre></div> <p>Save, click on the button, voilà!</p> <p>If it doesn't work, check that you have clicked on the play button of the viewer beforehand to load the model ;)</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 639px; " > <a class="gatsby-resp-image-link" href="/static/4b63325ad528660f2c6641def0da3ea5/738b8/change-background.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 71.35135135135134%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAAAsSAAALEgHS3X78AAAC/ElEQVQ4y21Ty05TURTdpxTQmKAENDFOjEkHDkgc6ECdOdE4IRLjhMQfMM6daGJinKA2cWLQpE0A48yJQUBjqRHEPqBAue1tKS3Ql/RBX9fSlsLy7FOfCYObm+y91jp7nb0OxbwxeIe98Nv90GwagjYdIVsIEXsE+rAO95AbniEPPE+88D71Qn+pqx5jGMucZclljc2lTZD/mR+jNIpJmoSTnHCRC4u0iABpCFMYUYpindaxJv8RWlM1TfYYw1jmMPc1jSFii4DC1jAmaALudjeWxTLCIowNsYGUSCEjMsiJHPIijx0y8IMqsraleoxhLHPcZjemaAqbdjlhzBrDDM0gYA6oaRicoxxKVIIhDFRFFbvUwMxxH771+qVwVfaK6iDGMoe5szSLlD0FilvjWKAFxMwxpCmNbdqGIaepUQ01UQcI8BzVcOrKNVy+cBvFjoo6oCIqCsucaFsUPvIhY8+AktYkVmgFSXNCTWZIW3Wqoyma2Kd9JXjf8gLdFy/hvKUf6c6squ2ImsIyJyG5AQogZ8+B0tY0ghRE2pxGgQqoSksNsavEqh11zPbpGBh8CMvVAfSd60e2c1sJ1kVDYZnD3BCFkLfnDxasm1pWnWeWcOPuI9y88xi3Bh9g7Ox7Vd8Te9iVhx4o+Nty4pflirRRk3aatId9SfKd/Ix3ljdYPD0No3tdCbIYX0vlH8scJWWZlzJP84ia5YYp1VqKvPAGNVHqiqPctYFcjzz9xCryPauoHSpKsYa6P8Yy57+lcGy+0hdoZk1FIElJZEUWZbnpzLEQqiYD5SMZFHqjMkIGip1bMjY8WVZhOfBam/Y3NvrzVbylD61gm2SwTTLYJhlsUxIZkwy2nCJzOIHvXWvIy/xx2LnHGMYyx9XuUq9FBdttC+Le9U8Yp3E4yIE5mlO55HvVSVdPLUKRP89OlwvkHmMYyxzmjtAIwq/CoFK8jJgzha3pLWQdWeQdeRQcBRQdRZQcJZQd5dY33fpzjXuMYSxzmJv4mICRMvATG50mMyr6bPMAAAAASUVORK5CYII='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="The viewer with the background changed" title="The viewer with the background changed" src="/static/4b63325ad528660f2c6641def0da3ea5/738b8/change-background.png" srcset="/static/4b63325ad528660f2c6641def0da3ea5/1d79a/change-background.png 185w, /static/4b63325ad528660f2c6641def0da3ea5/1efb2/change-background.png 370w, /static/4b63325ad528660f2c6641def0da3ea5/738b8/change-background.png 639w" sizes="(max-width: 639px) 100vw, 639px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <h2>Change the chair color</h2> <p>Alright, that function is a bit bigger but you'll have to trust me.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">function</span> <span class="token function">App</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token comment">// ...</span> <span class="token keyword">const</span> <span class="token function-variable function">changeChairColor</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> apiRef<span class="token punctuation">.</span>current<span class="token punctuation">.</span><span class="token function">getMaterialList</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token parameter">err<span class="token punctuation">,</span> materials</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">const</span> plasticMaterial <span class="token operator">=</span> materials<span class="token punctuation">.</span><span class="token function">find</span><span class="token punctuation">(</span> <span class="token punctuation">(</span><span class="token parameter">material</span><span class="token punctuation">)</span> <span class="token operator">=></span> material<span class="token punctuation">.</span>name <span class="token operator">===</span> <span class="token string">"Blue plastic"</span> <span class="token punctuation">)</span><span class="token punctuation">;</span> plasticMaterial<span class="token punctuation">.</span>channels<span class="token punctuation">.</span>AlbedoPBR<span class="token punctuation">.</span>color <span class="token operator">=</span> <span class="token punctuation">[</span> Math<span class="token punctuation">.</span><span class="token function">random</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> Math<span class="token punctuation">.</span><span class="token function">random</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> Math<span class="token punctuation">.</span><span class="token function">random</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">]</span><span class="token punctuation">;</span> apiRef<span class="token punctuation">.</span>current<span class="token punctuation">.</span><span class="token function">setMaterial</span><span class="token punctuation">(</span>plasticMaterial<span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span><span class="token string">"Updated chair color"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">;</span> <span class="token keyword">return</span> <span class="token punctuation">(</span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span> <span class="token attr-name">className</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>App<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>button</span> <span class="token attr-name">onClick</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>changeBackgroundColor<span class="token punctuation">}</span></span><span class="token punctuation">></span></span><span class="token plain-text">Change background color</span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>button</span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>button</span> <span class="token attr-name">onClick</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>changeChairColor<span class="token punctuation">}</span></span><span class="token punctuation">></span></span><span class="token plain-text">Change chair color</span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>button</span><span class="token punctuation">></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">Viewer</span></span> <span class="token attr-name">apiRef</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>apiRef<span class="token punctuation">}</span></span> <span class="token punctuation">/></span></span><span class="token plain-text"> </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>div</span><span class="token punctuation">></span></span> <span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span></code></pre></div> <p>A new color for our chair!</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 645px; " > <a class="gatsby-resp-image-link" href="/static/865a48ba10342a540cf23944f0e03fca/af192/change-chair.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 72.97297297297297%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAPCAYAAADkmO9VAAAACXBIWXMAAAsSAAALEgHS3X78AAADlklEQVQ4y02T60+aZxiH+zct2YclS5c11TlP9QAyFBRBdCDOAh5AEIR5ili1vkURRHSKWg9VQatuLlv2tdvarvu0pI0VRdEu+w927Xlf6LY3uZPneP1+930/761s5ozryxxXIi6zV0pcZcX4IsdvL37nl2fP+fXnF7x8/orTN+Ls1Q257DW5y5t/Q16T72cy59xa+ekQaecRsVSYlaMIyW+jJE8iLJ48JHY4jrQ7SDg1zOz+KImTKRLfTRE/miR28IBIKkRkJ8REMsj2yQryd6s9KaEfUWEZ13JfasQaVmGN1GKOlGGZr6R9oQprvFqMq2iLVmCeLcMoldA4UUz96B20g3cocX6AK2yBvwVQG7hNe7iWjqia+wkNzhUtPes63JsG+rYNeLab8ab1ePd1Yt6E63ETPat6HN/U0xnXYIuo0Y0WMZJ05R3qQ5/REROwxS/oWm3AtdmI54kB724z/SkjvgMBDQ3idkn4Nq149ppwbzUpoo5lkdWCBsNkCWMbfXmg8WEp9kXtf7AdA/1pI/60mYHjFrzJDmxlS+g+msYdceI/lsWMilsZ6lxuwBwu58ETbx4oT7pWGuh9rFecya78By34nhrx/qDGOzxG2915yj90Yp/sIPi9CV/KpGTg3mqkZ01P21wlU3u+PLB1poLuVZ3iTj7kS5vwHxoJ7tqJJQ5ZD/5FyPQSc3Ecd8xB4ESUId2iCMsue9cbscTuMZXyFxzOCIdJke5GPl1Z3XdkILjhYi7wjJjnFZL3R8akCQLHrfiftuDfN9G/JxwqQD1fRoXDVMGhSSpTOibXQ1b07jUrDvzHzQRWavBKGoKi8INrar4+EGIHeXeyuGtDT3dSR2ukgsnd/jzQMPE5neKC3LH3UI+ABkQJAqLzPXE1A3N19AnRYbnO6QJM7MuNdCxpMU2XMr7lyQMbhj6hfVbFV/E64VRA13R0izS6IjU0DhRR1fkxRvtttIEiHFGV0rxeIdy1rKFTvFvbnAp9qIjRtcI7rB5SYRwXf8B0JZZINR1xAV9Qi7+lhuZQKfrgXcxDxZgmy7HN12Jf0ijPzJpowTJTg1VSoQl8SjDhLACjj2geqcYcuodlWoVtpo7OqBZHvB5nokEJh4iuRfGQF8R4vkGUwSjW7NimtdglA/qBMgbjBYen53+QyZ1yfv2W85u3XNyccfHujOyf7yPzv8jvXYhz59evyVyfcnb1htPsa3LvLhXgPyMqKeUmXQLXAAAAAElFTkSuQmCC'); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="The viewer with the chair color changed" title="The viewer with the chair color changed" src="/static/865a48ba10342a540cf23944f0e03fca/af192/change-chair.png" srcset="/static/865a48ba10342a540cf23944f0e03fca/1d79a/change-chair.png 185w, /static/865a48ba10342a540cf23944f0e03fca/1efb2/change-chair.png 370w, /static/865a48ba10342a540cf23944f0e03fca/af192/change-chair.png 645w" sizes="(max-width: 645px) 100vw, 645px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <h2>Hooks refactor</h2> <p>To abstract the initialization code and clean up our component, we'll write a <a href="https://reactjs.org/docs/hooks-custom.html">custom hook</a> that leverages all the code previously written plus a <code>useState</code> for the api object.</p> <p>Custom hooks are one of the most powerful tools to keep our code well organized. Once you start using them, there's no going back.</p> <div class="gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">import</span> React<span class="token punctuation">,</span> <span class="token punctuation">{</span> useEffect<span class="token punctuation">,</span> useRef<span class="token punctuation">,</span> useState <span class="token punctuation">}</span> <span class="token keyword">from</span> <span class="token string">"react"</span><span class="token punctuation">;</span> <span class="token comment">// Our wonderful chair model</span> <span class="token keyword">const</span> <span class="token constant">MODEL_UID</span> <span class="token operator">=</span> <span class="token string">"c632823b6c204797bd9b95dbd9f53a06"</span><span class="token punctuation">;</span> <span class="token keyword">const</span> <span class="token function-variable function">useSketchfabViewer</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token comment">// This ref will contain the actual iframe object</span> <span class="token keyword">const</span> viewerIframeRef <span class="token operator">=</span> <span class="token function">useRef</span><span class="token punctuation">(</span><span class="token keyword">null</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">const</span> <span class="token punctuation">[</span>api<span class="token punctuation">,</span> setApi<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token function">useState</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">const</span> ViewerIframe <span class="token operator">=</span> <span class="token punctuation">(</span> <span class="token operator">&lt;</span>iframe <span class="token comment">// We feed the ref to the iframe component to get the underlying DOM object</span> ref<span class="token operator">=</span><span class="token punctuation">{</span>viewerIframeRef<span class="token punctuation">}</span> title<span class="token operator">=</span><span class="token string">"sketchfab-viewer"</span> style<span class="token operator">=</span><span class="token punctuation">{</span><span class="token punctuation">{</span> height<span class="token operator">:</span> <span class="token number">400</span><span class="token punctuation">,</span> width<span class="token operator">:</span> <span class="token number">600</span> <span class="token punctuation">}</span><span class="token punctuation">}</span> <span class="token operator">/</span><span class="token operator">></span> <span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">useEffect</span><span class="token punctuation">(</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token comment">// Initialize the viewer</span> <span class="token keyword">let</span> client <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">window<span class="token punctuation">.</span>Sketchfab</span><span class="token punctuation">(</span>viewerIframeRef<span class="token punctuation">.</span>current<span class="token punctuation">)</span><span class="token punctuation">;</span> client<span class="token punctuation">.</span><span class="token function">init</span><span class="token punctuation">(</span><span class="token constant">MODEL_UID</span><span class="token punctuation">,</span> <span class="token punctuation">{</span> success<span class="token operator">:</span> setApi<span class="token punctuation">,</span> <span class="token function-variable function">error</span><span class="token operator">:</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span><span class="token string">"Viewer error"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token comment">// We only want to initialize the viewer on first load</span> <span class="token comment">// so we don't add any dependencies to useEffect</span> <span class="token punctuation">[</span><span class="token punctuation">]</span> <span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">return</span> <span class="token punctuation">[</span>ViewerIframe<span class="token punctuation">,</span> api<span class="token punctuation">]</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">;</span> <span class="token keyword">export</span> <span class="token keyword">const</span> <span class="token function-variable function">Viewer</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token parameter"><span class="token punctuation">{</span> apiRef <span class="token punctuation">}</span></span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">const</span> <span class="token punctuation">[</span>ViewerIframe<span class="token punctuation">,</span> api<span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token function">useSketchfabViewer</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> apiRef<span class="token punctuation">.</span>current <span class="token operator">=</span> api<span class="token punctuation">;</span> <span class="token keyword">return</span> ViewerIframe<span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">;</span></code></pre></div> <h1>What we learned</h1> <ul> <li>We can use <strong>refs</strong> to feed <strong>DOM elements</strong> to libraries that need them.</li> <li>We can use <strong>refs to give back objects to parents</strong>. Another solution could have been to use a React Context.</li> <li>We can use <strong>custom hooks</strong> to make our code <strong>cleaner</strong> and more readable.</li> </ul> <p>You can find the final source code <a href="https://github.com/aldebout/sketchfab-react/tree/part1">here</a>.</p> <p><em>Shout-out to Sketchfab for their amazing product and in particular to <a href="https://github.com/arthurjamain">Arthur Jamain</a> for the great collaboration we had over the last few months.</em></p><![CDATA[Can I Use Web Components]]>/2020/04/can_i_use_web_components//2020/04/can_i_use_web_components/Fri, 10 Apr 2020 00:00:00 GMT<style> .quote { background-color: aliceblue; padding: 1em; margin: 1em 0; font-style: italic; } .quote span { font-style: normal; font-weight: bold; } .note { background-color: lightgray; padding: 0.5em; margin: 1em 0; border-left: 6px black solid; } </style> <p>The goal of this article is to show you what is a web component, why you can now use it in production, and when you should do so.</p> <p>For all those who already have some knowledge in web components, I would like to specify that I will be talking about the web component V1 standards as the v0 draft APIs are now removed from Chrome since Chrome 80.</p> <p>If you’re new to web components, don't worry you can catch up starting from the v1 standards, looking at the v0 specifications and standards might not be useful unless you want to understand the journey of the web components to reach its V1.</p> <p>But first, you might want to be able to understand the basic concepts of a web component.</p> <h2>Definition of a web component</h2> <p>So first and foremost, what is a web component?</p> <div class="quote"> "Web Components is a suite of different technologies allowing you to create reusable custom elements — with their functionality encapsulated away from the rest of your code — and utilize them in your web apps." - <a href="https://developer.mozilla.org/en-US/docs/Web/Web_Components" ><span>MDN</span></a> </div> <p>Simply put, the web component is not just a thing by itself. For a component to be called so, different technologies have to be combined.</p> <p>What are those "different technologies" then?</p> <p>I will go through the different technologies that make a web component. This article won't be able to encompass all the possibilities of a web component and all its possible implementations. For those who would like to go further into the explanations, I will provide some resources as you read through.</p> <h3>Custom elements</h3> <p>A web component is nothing more than an enhanced custom element. A custom element is a javascript class expanding the HTMLElement class that allows you to create a new HTML tag for your component.</p> <p>First, you have to create a class as shown in the script below.</p> <div class="gatsby-highlight" data-language="js"><pre class="language-js"><code class="language-js"><span class="token keyword">class</span> <span class="token class-name">MyComponent</span> <span class="token keyword">extends</span> <span class="token class-name">HTMLElement</span> <span class="token punctuation">{</span> <span class="token function">constructor</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">super</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token operator">...</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span></code></pre></div> <p>We then define a custom element with the previously created class.</p> <div class="gatsby-highlight" data-language="js"><pre class="language-js"><code class="language-js">customElements<span class="token punctuation">.</span><span class="token function">define</span><span class="token punctuation">(</span><span class="token string">"my-component"</span><span class="token punctuation">,</span> MyComponent<span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre></div> <p>Your custom element is ready to be used as a tag in any web project.</p> <div class="gatsby-highlight" data-language="html"><pre class="language-html"><code class="language-html"><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>my-component</span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>my-component</span><span class="token punctuation">></span></span></code></pre></div> <p>let's try it in a simple html document:</p> <div class="gatsby-highlight" data-language="html"><pre class="language-html"><code class="language-html"><span class="token doctype"><span class="token punctuation">&lt;!</span><span class="token doctype-tag">DOCTYPE</span> <span class="token name">html</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>html</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>head</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>meta</span> <span class="token attr-name">charset</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>utf-8<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>title</span><span class="token punctuation">></span></span>my first custom element<span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>title</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>head</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>body</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>my-component</span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>my-component</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>script</span><span class="token punctuation">></span></span><span class="token script"><span class="token language-javascript"> <span class="token keyword">class</span> <span class="token class-name">MyComponent</span> <span class="token keyword">extends</span> <span class="token class-name">HTMLElement</span> <span class="token punctuation">{</span> <span class="token function">constructor</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">super</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> customElements<span class="token punctuation">.</span><span class="token function">define</span><span class="token punctuation">(</span><span class="token string">"my-component"</span><span class="token punctuation">,</span> MyComponent<span class="token punctuation">)</span><span class="token punctuation">;</span> </span></span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>script</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>body</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>html</span><span class="token punctuation">></span></span></code></pre></div> <p>And that's it! you have created a custom element. Now if you inspect your HTML document on a web browser you should be able to see something like this:</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 544px; " > <span class="gatsby-resp-image-background-image" style="padding-bottom: 40%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAICAYAAAD5nd/tAAAACXBIWXMAABYlAAAWJQFJUiTwAAABZElEQVQoz42S21LiQBCG8yJLzhByFBQSSCYTQARh0WV3PZUWl77/G3x2cA9qlbV78VVPz0z/PdPdRn8wIinnxBON3+3h2NYrlilYuK6L53l/7L8wkuyErGxI6yWJiIbZ4CieSpJk2kiS7q/L/hvrv/E/CIb9gDivSJo1iTqnH0V4vYBulOKHCbZpYstrf1ur80XoYJkd2RMrv2gx5dyybYxJUVBUNeXmG+Vqx2iqGKsZo0rLumJcqlfEH4ufz5bkzYJcLxjXc3KJbzW01hRFLoLyXVU36N0t6usN+vqB+f0zev9E8+NAvbuj2T/SfD9QLneoq3uU7KntT8r1HjVboecXLC62VJLEcLoJ8XAiNdwQq0thQ3haEQ0K+mc1wakimp4TDEscP8bxwr/4EW18i92e9VKMtpDh2YRMapjpFUE2lC6b0lUH17Hf4XkfO+2+RybBCNLBsZtptaAdIddx/ms8PuMFS2H3b/LPxUAAAAAASUVORK5CYII='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="inspect my first web component" title="inspect my first web component" src="/static/5012801d71a4cfeb120a802690c64ad2/b3e51/inpect-my-component-v1.png" srcset="/static/5012801d71a4cfeb120a802690c64ad2/1d79a/inpect-my-component-v1.png 185w, /static/5012801d71a4cfeb120a802690c64ad2/1efb2/inpect-my-component-v1.png 370w, /static/5012801d71a4cfeb120a802690c64ad2/b3e51/inpect-my-component-v1.png 544w" sizes="(max-width: 544px) 100vw, 544px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </span></p> <p>The main idea of custom elements is that you can create, extend, and reuse them everywhere in your code.</p> <p>If you are looking for a more in-depth documentation, check <a href="https://html.spec.whatwg.org/multipage/custom-elements.html">the official documentation</a>. I would also recommend <a href="https://developers.google.com/web/fundamentals/web-components/customelements">this article on customElements</a> that is giving you a good overview of the matter.</p> <h3>Shadow DOM</h3> <p>The next technology that we will be discussing is the Shadow DOM. When you attach a shadow DOM to your custom element you then scope all the CSS, and the HTML in this DOM. The big idea of the shadow DOM is to be able to create templates for your web components that won't be affected by any style or js selectors and therefore keep your code and feature as intended.</p> <p>Let's try it in our previously created component :</p> <p>First create a shadow root:</p> <div class="gatsby-highlight" data-language="js"><pre class="language-js"><code class="language-js"><span class="token keyword">const</span> shadowRoot <span class="token operator">=</span> <span class="token keyword">this</span><span class="token punctuation">.</span><span class="token function">attachShadow</span><span class="token punctuation">(</span><span class="token punctuation">{</span> mode<span class="token operator">:</span> <span class="token string">"open"</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre></div> <p>If you want to know more about the attachShadow mode argument I strongly recommend this <a href="https://blog.revillweb.com/open-vs-closed-shadow-dom-9f3d7427d1af">article</a> from Leon Revill.</p> <p>Create some elements:</p> <div class="gatsby-highlight" data-language="js"><pre class="language-js"><code class="language-js"><span class="token keyword">const</span> spanElement <span class="token operator">=</span> document<span class="token punctuation">.</span><span class="token function">createElement</span><span class="token punctuation">(</span><span class="token string">"span"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> spanElement<span class="token punctuation">.</span>textContent <span class="token operator">=</span> <span class="token string">"Hello world"</span><span class="token punctuation">;</span></code></pre></div> <p>Create a style element:</p> <div class="gatsby-highlight" data-language="js"><pre class="language-js"><code class="language-js"><span class="token keyword">const</span> style <span class="token operator">=</span> document<span class="token punctuation">.</span><span class="token function">createElement</span><span class="token punctuation">(</span><span class="token string">"style"</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre></div> <p>Create a css class for your span element and add some css rules:</p> <div class="gatsby-highlight" data-language="js"><pre class="language-js"><code class="language-js">style<span class="token punctuation">.</span>textContent <span class="token operator">=</span> <span class="token string">".my-class {color: red}"</span><span class="token punctuation">;</span> spanElement<span class="token punctuation">.</span><span class="token function">setAttribute</span><span class="token punctuation">(</span><span class="token string">"class"</span><span class="token punctuation">,</span> <span class="token string">"my-class"</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre></div> <p>And finally attach them to your shadow root:</p> <div class="gatsby-highlight" data-language="js"><pre class="language-js"><code class="language-js">shadowRoot<span class="token punctuation">.</span><span class="token function">appendChild</span><span class="token punctuation">(</span>style<span class="token punctuation">)</span><span class="token punctuation">;</span> shadowRoot<span class="token punctuation">.</span><span class="token function">appendChild</span><span class="token punctuation">(</span>spanElement<span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre></div> <p>so here it what your component should look like now:</p> <div class="gatsby-highlight" data-language="html"><pre class="language-html"><code class="language-html"><span class="token doctype"><span class="token punctuation">&lt;!</span><span class="token doctype-tag">DOCTYPE</span> <span class="token name">html</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>html</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>head</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>meta</span> <span class="token attr-name">charset</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>utf-8<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>title</span><span class="token punctuation">></span></span>my first custom element<span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>title</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>head</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>body</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>my-component</span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>my-component</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>script</span><span class="token punctuation">></span></span><span class="token script"><span class="token language-javascript"> <span class="token keyword">class</span> <span class="token class-name">MyComponent</span> <span class="token keyword">extends</span> <span class="token class-name">HTMLElement</span> <span class="token punctuation">{</span> <span class="token function">constructor</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">super</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">const</span> shadowRoot <span class="token operator">=</span> <span class="token keyword">this</span><span class="token punctuation">.</span><span class="token function">attachShadow</span><span class="token punctuation">(</span><span class="token punctuation">{</span> mode<span class="token operator">:</span> <span class="token string">"open"</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">const</span> spanElement <span class="token operator">=</span> document<span class="token punctuation">.</span><span class="token function">createElement</span><span class="token punctuation">(</span><span class="token string">"span"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> spanElement<span class="token punctuation">.</span>textContent <span class="token operator">=</span> <span class="token string">"Hello world"</span><span class="token punctuation">;</span> spanElement<span class="token punctuation">.</span><span class="token function">setAttribute</span><span class="token punctuation">(</span><span class="token string">"class"</span><span class="token punctuation">,</span> <span class="token string">"my-class"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">const</span> style <span class="token operator">=</span> document<span class="token punctuation">.</span><span class="token function">createElement</span><span class="token punctuation">(</span><span class="token string">"style"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> style<span class="token punctuation">.</span>textContent <span class="token operator">=</span> <span class="token string">".my-class {color: red;}"</span><span class="token punctuation">;</span> shadowRoot<span class="token punctuation">.</span><span class="token function">appendChild</span><span class="token punctuation">(</span>style<span class="token punctuation">)</span><span class="token punctuation">;</span> shadowRoot<span class="token punctuation">.</span><span class="token function">appendChild</span><span class="token punctuation">(</span>spanElement<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> customElements<span class="token punctuation">.</span><span class="token function">define</span><span class="token punctuation">(</span><span class="token string">"my-component"</span><span class="token punctuation">,</span> MyComponent<span class="token punctuation">)</span><span class="token punctuation">;</span> </span></span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>script</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>body</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>html</span><span class="token punctuation">></span></span></code></pre></div> <p>Now you can try going to the console and access and modify the elements in the shadow DOM but you won't be able to.</p> <p><img src="/6cbaffa01367a59ee123ec4ef27f72b0/shadowDOMInspect.gif" alt="inspect my first web component with shadow root"></p> <p>As well if you add some style in your HTML file, it won't affect the shadow DOM elements. Try adding style on the span element for example:</p> <div class="gatsby-highlight" data-language="html"><pre class="language-html"><code class="language-html"><span class="token doctype"><span class="token punctuation">&lt;!</span><span class="token doctype-tag">DOCTYPE</span> <span class="token name">html</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>html</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>head</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>meta</span> <span class="token attr-name">charset</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>utf-8<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>title</span><span class="token punctuation">></span></span>my first custom element<span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>title</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>style</span><span class="token punctuation">></span></span><span class="token style"><span class="token language-css"> <span class="token selector">span</span> <span class="token punctuation">{</span> <span class="token property">color</span><span class="token punctuation">:</span> blue<span class="token punctuation">;</span> <span class="token punctuation">}</span> </span></span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>style</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>head</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>body</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>span</span><span class="token punctuation">></span></span>Hey! I am just here for the example tho...<span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>span</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>my-component</span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>my-component</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>script</span><span class="token punctuation">></span></span><span class="token script"><span class="token language-javascript"> <span class="token punctuation">{</span><span class="token operator">...</span><span class="token punctuation">}</span> </span></span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>script</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>body</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>html</span><span class="token punctuation">></span></span></code></pre></div> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 738px; " > <span class="gatsby-resp-image-background-image" style="padding-bottom: 9.189189189189188%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAACCAYAAABYBvyLAAAACXBIWXMAABYlAAAWJQFJUiTwAAAAbElEQVQI1y2NyxKDMAwD+f9fbAl5NiEwOcChvdRbu3DQjDwrS1OtsG2QEjT16wrOQX2p9E5ReYecwS9QihADfN5fDc7I4hALeA+9M7UGY0Dwd2m7mBV3VcnXoPHnw7iQNXceghnRp3+hrew7P+pIl4DenbhVAAAAAElFTkSuQmCC'); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="try to apply style on shadow DOM" title="try to apply style on shadow DOM" src="/static/3ad774d3b17e0e923831060759249497/774b6/shadowDOMStyle.png" srcset="/static/3ad774d3b17e0e923831060759249497/1d79a/shadowDOMStyle.png 185w, /static/3ad774d3b17e0e923831060759249497/1efb2/shadowDOMStyle.png 370w, /static/3ad774d3b17e0e923831060759249497/774b6/shadowDOMStyle.png 738w" sizes="(max-width: 738px) 100vw, 738px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </span></p> <p>As you can see the span inside the shadow DOM is not affected by the style.</p> <h3>HTML Template</h3> <div class="quote"> "The template element is used to declare fragments of HTML that can be cloned and inserted in the document by script. In a rendering, the template element represents nothing. The template contents of a template element are not children of the element itself." - <span>html.spec.whatwg.org</span> </div> <p>This means that you can define a template for your Web component and clone it for every instance of your component.</p> <p>So first you create your template:</p> <div class="gatsby-highlight" data-language="js"><pre class="language-js"><code class="language-js"><span class="token keyword">const</span> template <span class="token operator">=</span> document<span class="token punctuation">.</span><span class="token function">createElement</span><span class="token punctuation">(</span><span class="token string">"template"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> template<span class="token punctuation">.</span>innerHTML <span class="token operator">=</span> <span class="token template-string"><span class="token template-punctuation string">`</span><span class="token string"> &lt;style> span { color: red; } &lt;/style> &lt;span>Hey There! I am in the template &lt;/p> </span><span class="token template-punctuation string">`</span></span><span class="token punctuation">;</span></code></pre></div> <p>Then you can clone it's content into your shadow tree:</p> <div class="gatsby-highlight" data-language="js"><pre class="language-js"><code class="language-js">shadowRoot<span class="token punctuation">.</span><span class="token function">appendChild</span><span class="token punctuation">(</span>template<span class="token punctuation">.</span>content<span class="token punctuation">.</span><span class="token function">cloneNode</span><span class="token punctuation">(</span><span class="token boolean">true</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre></div> <p>As I am sure you noticed, we don't just add the template in the shadowRoot but instead we copy its content. That's because Templates are not rendered so what we need for our component is the content of the template.</p> <p>Let's implement it in our component:</p> <div class="gatsby-highlight" data-language="js"><pre class="language-js"><code class="language-js"><span class="token keyword">export</span> <span class="token keyword">class</span> <span class="token class-name">MyComponent</span> <span class="token keyword">extends</span> <span class="token class-name">HTMLElement</span> <span class="token punctuation">{</span> <span class="token function">constructor</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">super</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">const</span> shadowRoot <span class="token operator">=</span> <span class="token keyword">this</span><span class="token punctuation">.</span><span class="token function">attachShadow</span><span class="token punctuation">(</span><span class="token punctuation">{</span> mode<span class="token operator">:</span> <span class="token string">"open"</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">const</span> template <span class="token operator">=</span> document<span class="token punctuation">.</span><span class="token function">createElement</span><span class="token punctuation">(</span><span class="token string">"template"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> template<span class="token punctuation">.</span>innerHTML <span class="token operator">=</span> <span class="token template-string"><span class="token template-punctuation string">`</span><span class="token string"> &lt;style> span { color: red; } &lt;/style> &lt;span>Hey There! I am in the template &lt;/span></span><span class="token template-punctuation string">`</span></span><span class="token punctuation">;</span> shadowRoot<span class="token punctuation">.</span><span class="token function">appendChild</span><span class="token punctuation">(</span>template<span class="token punctuation">.</span>content<span class="token punctuation">.</span><span class="token function">cloneNode</span><span class="token punctuation">(</span><span class="token boolean">true</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span></code></pre></div> <p>The important point to keep in mind about templates is that it allows you to declare chunks of HTML in your document that will be rendered and that sole purpose is to be cloned and reused. It is a perfect fit for web components.</p> <p>You could define it in your <code>index.html</code> since template elements are not rendered in the browser but I would recommend keeping it close to your component declaration to keep your <code>index.html</code> file clean and organize your templates by component.</p> <p>You can even use the slot Element to make your template more flexible. Those of you who already worked with Vue.js may be familiar with the concept of slot. For the others Slots are like placeholder. They allow the developper to create specific elements that will be placed inside the parents component DOM at predifined emplacement.</p> <p>Imagine that your component template is a text with holes, each hole has an id. When you add children to your component, you can assign them a <code>slot</code> attribute. This slot attribute allows your component to place them in the "holes " according to their ids (the slot attributes).</p> <p>If you want to learn more about slot element with web component I would recommend this <a href="https://developer.mozilla.org/en-US/docs/Web/Web_Components/Using_templates_and_slots">documentation</a> from MDN.</p> <p>The addition of those four standards-based technologies is what defines the web component today.</p> <h3>ES Modules</h3> <p>This part is not always explicit in the definition of a web component. It doesn't affect the component on itself but more the way you will be able to use it in a project.</p> <div class="quote"> "The ES Modules specification defines the inclusion and reuse of JS documents in a standards-based, modular, performant way." - <span>webcomponents.org</span> </div> <p>The ES modules should allow developers to use the script tag with the type module. Wich means you can export variables or components from your <code>.mjs</code> files and import it in your <code>.html</code> file within a module script tag.</p> <p>The ES modules standard allow developers to reuse web component in any HTML file by importing them from an mjs file.</p> <div class="gatsby-highlight" data-language="js"><pre class="language-js"><code class="language-js"><span class="token comment">// wc.mjs</span> <span class="token keyword">export</span> <span class="token keyword">class</span> <span class="token class-name">MyComponent</span> <span class="token keyword">extends</span> <span class="token class-name">HTMLElement</span> <span class="token punctuation">{</span> <span class="token function">constructor</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">super</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">const</span> shadowRoot <span class="token operator">=</span> <span class="token keyword">this</span><span class="token punctuation">.</span><span class="token function">attachShadow</span><span class="token punctuation">(</span><span class="token punctuation">{</span> mode<span class="token operator">:</span> <span class="token string">"open"</span> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">const</span> spanElement <span class="token operator">=</span> document<span class="token punctuation">.</span><span class="token function">createElement</span><span class="token punctuation">(</span><span class="token string">"span"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> spanElement<span class="token punctuation">.</span>textContent <span class="token operator">=</span> <span class="token string">"Hello world"</span><span class="token punctuation">;</span> spanElement<span class="token punctuation">.</span><span class="token function">setAttribute</span><span class="token punctuation">(</span><span class="token string">"class"</span><span class="token punctuation">,</span> <span class="token string">"my-class"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">const</span> style <span class="token operator">=</span> document<span class="token punctuation">.</span><span class="token function">createElement</span><span class="token punctuation">(</span><span class="token string">"style"</span><span class="token punctuation">)</span><span class="token punctuation">;</span> style<span class="token punctuation">.</span>textContent <span class="token operator">=</span> <span class="token string">".my-class {color: red;}"</span><span class="token punctuation">;</span> shadowRoot<span class="token punctuation">.</span><span class="token function">appendChild</span><span class="token punctuation">(</span>style<span class="token punctuation">)</span><span class="token punctuation">;</span> shadowRoot<span class="token punctuation">.</span><span class="token function">appendChild</span><span class="token punctuation">(</span>spanElement<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span></code></pre></div> <div class="gatsby-highlight" data-language="html"><pre class="language-html"><code class="language-html"><span class="token doctype"><span class="token punctuation">&lt;!</span><span class="token doctype-tag">DOCTYPE</span> <span class="token name">html</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>html</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>head</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>meta</span> <span class="token attr-name">charset</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>utf-8<span class="token punctuation">"</span></span> <span class="token punctuation">/></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>title</span><span class="token punctuation">></span></span>my first custom element<span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>title</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>style</span><span class="token punctuation">></span></span><span class="token style"><span class="token language-css"> <span class="token selector">span</span> <span class="token punctuation">{</span> <span class="token property">color</span><span class="token punctuation">:</span> blue<span class="token punctuation">;</span> <span class="token punctuation">}</span> </span></span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>style</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>head</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>body</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>span</span><span class="token punctuation">></span></span>Hey! i am just here for the example tho...<span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>span</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>my-component</span><span class="token punctuation">></span></span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>my-component</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>script</span> <span class="token attr-name">type</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>module<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token script"><span class="token language-javascript"> <span class="token keyword">import</span> <span class="token punctuation">{</span> MyComponent <span class="token punctuation">}</span> <span class="token keyword">from</span> <span class="token string">"./wc.mjs"</span><span class="token punctuation">;</span> customElements<span class="token punctuation">.</span><span class="token function">define</span><span class="token punctuation">(</span><span class="token string">"my-component"</span><span class="token punctuation">,</span> MyComponent<span class="token punctuation">)</span><span class="token punctuation">;</span> </span></span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>script</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>body</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>html</span><span class="token punctuation">></span></span></code></pre></div> <div class="note"> An MJS file is a source code file containing an ES Module. </div> <div class="note"> You will need a server to be able to fetch with import, as it doesn’t work on the file:// protocol. You can use npx serve to start up a server in the current directory for testing locally. </div> <p>If you are interested in how the ES module works I strongly advise that you read this <a href="https://hacks.mozilla.org/2018/03/es-modules-a-cartoon-deep-dive/">article</a> from Lin Clark.</p> <h2>Can I use it?</h2> <p>Now, a lot of developers are still quite reluctant to use web components in their projects. And you can understand why! The lack of common standards implemented by all major browsers since it's creation in the early 2010s has allowed many web component libraries to raise and grow sub-communities that do not share the same opinion on the definition of a web component and its implementation.</p> <p>On the article <a href="https://dev.to/ben/why-the-react-community-is-missing-the-point-about-web-components-1ic3">Why the React community is missing the point about Web Components</a> by Ben Halpern in November 2018, Dan Abramov comments the following :</p> <div class="quote"> [...] The problem is that there's no single "web component community". There are multiple sub-communities. You mention "web component libraries" in the post. These libraries don't agree on a single standard so React would need to pick a side in debates like "how does server rendering work with web components". Whatever we pick will have large downstream effects, and our reluctance to support more has to do with being careful [...] about the semantics — not somehow being at odds with web components per se. </div> <p>So I think some important points have to be clarified for you to understand the implication of using it in your projects.</p> <h3>Web components are tools</h3> <p>It is important to understand that Web components are not designed to be better than HTML elements. We can consider web components as a tool to extend HTML, not to replace it.</p> <p>And as we know the same tool cannot be used for all projects. So before adding this tool to your toolbox make sure you really need it or else it might some unecessary weight.</p> <p>Here are some points that you should consider before jumping in :</p> <ol> <li>It is not that simple to write a web component. It can be quite complex and verbose depending on the scope of your web components. And most of the project with quite a large number of web components have been using some libraries to creates them. <a href="https://www.webcomponents.org/libraries">webcomponents.org</a> has a page dedicated to those libraries.</li> <li>Comparing a web component and a react component for example is like comparing apples and oranges. The definition of a web component is not setting limits to what you can put in it. React in its documentation explain it this way:</li> </ol> <div class="quote"> "React and Web Components are built to solve different problems. Web Components provide strong encapsulation for reusable components, while React provides a declarative library that keeps the DOM in sync with your data. The two goals are complementary. As a developer, you are free to use React in your Web Components or to use Web Components in React, or both." </div> <p>Understanding and defining what will be the interfaces between web components and the rest of your project's tools is important.</p> <ol start="3"> <li>Web component v1 standards are not that old and you probably want your project to work on older browsers versions too. Simply put, you might need polyfills. I will come back to it later on.</li> <li>One of the down points of the web component often cited is that developers have been struggling to use it with server-side rendering. This is mainly caused by the fact that the contents of ShadowDOM are often missing when the view is delivered to the client. One possibility is to remove the shadow dom from your component like you can see in <a href="https://lamplightdev.com/blog/2019/07/20/how-to-server-side-render-web-components/">this article</a>. Other web component advocates found ways to make it work like in <a href="https://dev.to/steveblue/server-side-rendering-web-components-320g">this article</a> by Steve Belovarich or <a href="https://www.petergoes.nl/blog/differentiate-ssr-states-in-a-web-component/">this one</a> by Peter Goes. I think you can sum it up by saying that for now there are no standards for server-side rendering with a web component but you can always find a workaround.</li> </ol> <p>Just remember to take all this point in account before jumping in.</p> <h3>Future proof</h3> <p>One of the main arguments of the pro web component community is that web components are future proof. Since it is based on web Standards, it is now supported by all modern browsers and should always be.</p> <p>I can already hear your ask:</p> <div class="quote"> "Alright that's for the future? But what about our beloved past heroes like IE?" </div> <p>Well you are right to be worried about them and that should be one of your main concerns.</p> <p>Below are the previously cited technologies browser supports at the current date. If you want the latest data, click on the link below each of them.</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <span class="gatsby-resp-image-background-image" style="padding-bottom: 29.72972972972973%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAGCAYAAADDl76dAAAACXBIWXMAABYlAAAWJQFJUiTwAAABoUlEQVQY0z2PTU8TYRSF559pIiv5Ay7UGKLukY0fialGrIZgQiCKmgKLimlCNKEYGQdLEWhF6dB25p1pZT46A21n2qYqrW4f325cPDm599ybk6P4tQM8q4hnfqEh9nDsHfrNMoO4xllkc9Y2JOZ/BpKfzSqxX2IQCX5LvxPq9E5NiYUSGVliSyXyDomCIyLnqzySz0GRYbDP326dYcfiT2wzjKV2RipkmJB7W86WDKpIX8gAE6VdyhBWN/DELg1dJQxtAv+IE3OTttBoHH+jKT3f2MYX25xU8vwwCviykVPeQpRyhI6NaxZwjR2Ubt/jVyCIMsuIG5dwlxO0CiotbY0g84zW4R6xV6Ina/bbFt2mpFGmN9o1ZCMZ5i094LSiEYdVFD81S3ruOk9eXGX+9QSPPlzkY2qM9JsLzGTH2V0cI9TXsTfT6CvTWKuPOa7lqDt5XHefsp0lsX4O/WCFbr2A8j0xyau3d5jM32Zi4ybXslMsaeO81K7wVEuyljtPcX6KdOoei+9u8en9ZZ6r08xtJVmQ/kgffk6iLtynuHqXf2Oli92IncXUAAAAAElFTkSuQmCC'); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="can i use custom element v1" title="can i use custom element v1" src="/static/6e0658c57d2df00a494fdcb7eb2ced6a/50383/can-i-use-custom-element-v1.png" srcset="/static/6e0658c57d2df00a494fdcb7eb2ced6a/1d79a/can-i-use-custom-element-v1.png 185w, /static/6e0658c57d2df00a494fdcb7eb2ced6a/1efb2/can-i-use-custom-element-v1.png 370w, /static/6e0658c57d2df00a494fdcb7eb2ced6a/50383/can-i-use-custom-element-v1.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </span> <a href="https://caniuse.com/#feat=custom-elementsv1">can i use custom element v1</a></p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <span class="gatsby-resp-image-background-image" style="padding-bottom: 37.2972972972973%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAHCAYAAAAIy204AAAACXBIWXMAABYlAAAWJQFJUiTwAAABxUlEQVQoz02RbWvTYBSG85P8G371g8J0A2EiwmQIMhGciFBRwReUMYYyQTZBVyerrpNZpd3Wpi9rl65pmrVp2iZZ0qT1bdZ9vfY0U/HDxbkPh3M/N+eR2toWreompsDW0vQaOQ48lZ9uJWSof3WrYR0yELrf2eGrrTDwNb65Cr5VErVGYKlIZjWNoW5h7KZC6uXkH1LsKUmalY1/xn/5/7FjvSso891RkFrldUxlPVx0RDrPyNM1t+kaGXxDxm8VORCpfge1MNEg0ITWwn7IodAD/3h+2NOReqVFPCWKLcxcQxi2Sngdha76np62ht8pC2Mxa2TxGjLuXha7ng97W09jCe11KuyLfUfMJM9RCWwNO/6a6vQE3XqRQM/Tr2UJqjKWKuPVMthmlv12Abe5jannsGvi7oLG5gdaK4sM/6KpbiC5yXekXkVYeniRt9MjfJy/zurcVWLPr/HpxRTll5N4TRkns0o7voCVXKbXKeKbBX44OqX8LAtvTorTFOhbO0jFK+PceTLK2fgFzqyMMRI9x6ml04wuj3MjNkY0doLcoymezUwQmT3P/flLPE3c5XEiwkziHrcTN7m8NsmDzxG+zN3iCJQX1Z0O3VjiAAAAAElFTkSuQmCC'); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="can i use custom shadow DOM v1" title="can i use custom shadow DOM v1" src="/static/a570e424471085894bec4e3c24ee9a59/50383/can-i-use-shadow-DOM-v1.png" srcset="/static/a570e424471085894bec4e3c24ee9a59/1d79a/can-i-use-shadow-DOM-v1.png 185w, /static/a570e424471085894bec4e3c24ee9a59/1efb2/can-i-use-shadow-DOM-v1.png 370w, /static/a570e424471085894bec4e3c24ee9a59/50383/can-i-use-shadow-DOM-v1.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </span> <a href="https://caniuse.com/#feat=shadowdomv1">can i use custom shadow DOM v1</a></p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <span class="gatsby-resp-image-background-image" style="padding-bottom: 34.054054054054056%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAHCAYAAAAIy204AAAACXBIWXMAABYlAAAWJQFJUiTwAAAB10lEQVQozzWQy28SYRTF+ft0YUiIJtWq0agrF8ZEFz6WaoxibVqVRKREY0wVbagF20IoM4VSkNcMMJShDAyUV0GgNHb58wN1cXIf37nn3vNZ9lWJSl6mrIapahGM3DaaEqRpxBk0FEZNlXErx9E/jNt5OkZSvGU5FvlhI0WnnqXXKNA1VSxlRaIqBKuFCBPxCdqVpBhK0SwnGB6oHHfyAoWp2CT+F57mIo7bKqODrOAqWAaFz3QnW8WWbj1Dr5WnZ0gclVYY/9IZttTppb1aanpV30zTF7y+Kbi19LQ/qYfNvzyL7llCW3xKI+GnshugGg1Qk32Y217KoTX2d9Yx1BAlbYt6TqKYldhTwpTTGygxMaP+FN+0i5bcEO7CWAKrc7x9dI7EvWvs3LmCvHAZ6c1ZEg9uEnRY0dMfGdaFNVXmcC/K2FQ4aRX53RS9QRn9hxtD/spoVGEonFru222c9s1yyWXjosCFL1ZmPGe44T6P1XsKt/MWfvtd5p/MsHXVxvL7h7iir/mwOYfyzs5t5yyPnddRHc/Qvi8JyykfcmGFhcRzHMmXvIq94FPSiaIHWIzPEwm6KEW/EdP91GUvmYyXSHGVeM6LGfKQ1dbJZ9cwN5dpCdt/AAnGz+gA+TAUAAAAAElFTkSuQmCC'); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="can i use javascript module script tag" title="can i use javascript module script tag" src="/static/85161127ac2e277165df695edf17bcdc/50383/can-i-use-js-module-script-tag.png" srcset="/static/85161127ac2e277165df695edf17bcdc/1d79a/can-i-use-js-module-script-tag.png 185w, /static/85161127ac2e277165df695edf17bcdc/1efb2/can-i-use-js-module-script-tag.png 370w, /static/85161127ac2e277165df695edf17bcdc/50383/can-i-use-js-module-script-tag.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </span> <a href="https://caniuse.com/#feat=es6-module">can i use javascript module script tag</a></p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <span class="gatsby-resp-image-background-image" style="padding-bottom: 28.64864864864865%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAGCAYAAADDl76dAAAACXBIWXMAABYlAAAWJQFJUiTwAAABmklEQVQY022Qy2sTURSH5x/UhVQKRRShghtXLor4QCn4iFLcGC3WpNpQHY0NVkKFhpp0krRN0jQz986kTWam1CSdmT6Cj8bt523EnYuPc/lxzuF8V3NlGV8auFYBV6ziyVV8u0jLKhJ4Nfpdk5+B/C8/9gWhV1U9ku+Ry1HooIXWIqFfJfzWUAsqRH6NoFUisrPDd79jMjhoMogcThSDg380+X24zUloq1wOs1+qapH9mW6rSmdjiZ5XJ+jYBO46R1aGaM+k16rQcwy8HYPOTunv9bLIrvjK9uYybWXSbZv4yu4UrV34RGPmAWvXRtlKxBD6M8T8U5rpl2zNxvCMDMc9waH6kn6jwLGzQX9PKfqbRCr3ylnkfJz9U0OFFrt3hgup84ynxhhPjnAlMcLlV+e4mhxlLHGW6TfXqSef8OLxJaYfXiQ5N4G+PsNcKU49/ZzU7MRwrv56CvEujmZWMmSlzs38De4ad7ht3OJR+T615hcm1ybRc1Ps5t6zIhcoiAXy1kfyZpoVkaaZe4sofsBwFnGXUvjLOn8AgcaKNPwKN2EAAAAASUVORK5CYII='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="can i use javascript module dynamic import" title="can i use javascript module dynamic import" src="/static/7b356ff7f7b1813d67136d8bd0759e58/50383/can-i-use-js-module-dynamic-import.png" srcset="/static/7b356ff7f7b1813d67136d8bd0759e58/1d79a/can-i-use-js-module-dynamic-import.png 185w, /static/7b356ff7f7b1813d67136d8bd0759e58/1efb2/can-i-use-js-module-dynamic-import.png 370w, /static/7b356ff7f7b1813d67136d8bd0759e58/50383/can-i-use-js-module-dynamic-import.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </span> <a href="https://caniuse.com/#feat=es6-module-dynamic-import">can i use javascript module dynamic import</a></p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <span class="gatsby-resp-image-background-image" style="padding-bottom: 34.5945945945946%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAHCAYAAAAIy204AAAACXBIWXMAABYlAAAWJQFJUiTwAAAB2UlEQVQozz2Py08TYRTF568zmmg0btSl7CRhYTVGI2k0kWhMjG1MRYhW8AWiAQlWHhJKSx8zfdLpdFro0BedzrTUVPvA5c+vaFycnHPul/Ode6VqQaashzHUAKW9ID/qu3TqKh1ziKzQGbqW9h89O3fy3ionT/TPhopVS9A2c7RqGtKhvk1V81PNhwUiNIwYdimBbci0DmSOKim6Ijho5ek3dfotnYHg3r/ZQPhuU6Nvi8JGFuko/QErPc/hXpR6IUzjIE7zUKWpr9LWlkVrRnwcwzIUbMF2UcE04sLHaBRlkZNFeUosInJiEck0EmKYxCymsCsxzGoB9ekEhVdurJJKJRvFzIWo7YsLjAh1oQ09SjkX5EDdopAW12lxStoOhhZA6v0qU9t6iTHnpBVcofhpnNT4OTKPLmMs3iM/66Bdj9HRwnTimwzsAr8tnWMrz/FQ20P+64dayj904pq6yOjCKZyeK9xZvsDN1fM4fGdxLJ3h+pfT7G548b67zX33Jb55brGtfMSfXGAn8Zmvyhucm3dZUWYJJReR9mdcrPk8PFFcjGxcY2z9Bld9ozzwT/A6Ms3I2hjR2cesf5/Em5jifcDNfESw/IK56JAnmQk94230OQvyNH8AftzQnYyzcO0AAAAASUVORK5CYII='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="can i use HTML templates" title="can i use HTML templates" src="/static/d93f281bcbd36c21f6105c7243c565a5/50383/can-i-use-HTML-templates.png" srcset="/static/d93f281bcbd36c21f6105c7243c565a5/1d79a/can-i-use-HTML-templates.png 185w, /static/d93f281bcbd36c21f6105c7243c565a5/1efb2/can-i-use-HTML-templates.png 370w, /static/d93f281bcbd36c21f6105c7243c565a5/50383/can-i-use-HTML-templates.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </span> <a href="https://caniuse.com/#feat=template">can i use HTML templates</a></p> <p>As you can see neither Internet explorer is not supporting any of them neither the early versions of other browsers.</p> <p>There are polyfills for the web component APIs. You can use <a href="https://github.com/webcomponents/polyfills/tree/master/packages/webcomponentsjs">the official polyfills</a>.</p> <p>With those polyfills, nothing you can now develop your future and past proof web component!</p> <h3>Reusability</h3> <p>A web component is easily reusable and is trustworthy. Its strong encapsulation provides security for its data. It also ensures that it will be used the way its designer intended to.</p> <h2>Should I use it?</h2> <p>Based on previous points, there are no specific reasons to not go for it. Still, the Web component isn't mature enough to be trusted for production by most developers. It solves problems but adds others.</p> <p>For now, the best use the community can agree on for a WC-based project is a design system UI components library. Tesla for example, has crossed the bridge and chosen to make its design system with web components.</p> <p>Building a sustainable design system can be a real brain teaser for companies. Big companies often use a lot of different technologies and frameworks for all their products.</p> <p>They reach a point where sustainability becomes a problem in terms of design and technology. They ask themselves those questions :</p> <ul> <li>Do we have to create a UI component library for each framework that we use?</li> <li>How can we be sure that we don’t switch framework in the next few years?</li> <li>And If we make sure we don't, aren't we missing an opportunity?</li> </ul> <p>A web component is not based on any framework owned by any company but on standards adopted by all major browsers. You don't have to worry about the hype effect on your chosen technology or framework. This is why web components fit if you decide to create a UI component Library or even a design system.</p> <p>Of course, you can also use it in any static pages project but I wanted to highlight the one big use where you should consider it.</p> <h2>Conclusion</h2> <p>Now that the standards that define Web components are implemented in every modern browser, they are meant to be part of the future of the web.</p> <p>As a web developer, it’s time to consider web components as a practical solution for your features and problems.</p> <p>I would like to quote <a href="https://twitter.com/serhiikulykov">Serhii Kulykov</a>(@webpadawan) on <a href="https://dev.to/webpadawan/the-journey-of-web-components-wrong-ways-lacking-parts-and-promising-paths-1d5a">this article</a></p> <div class="quote"> "Now in 2019, it’s time to give a try to Custom Elements and Shadow DOM. They might not be “the next big thing” by themselves, but rather solid foundation the next big thing could be built upon." - <span>Serhii Kulykov</span> </div> <p>Of course when the technology will be more mature, accepted and integrated into our projects we will witness the apparition of new ideas and maybe even new standards.</p> <p>But for now, we are the ones who have to be bold enough to try on our own.</p> <p>If you are interested in the subject, I recommend the following articles:</p> <p><a href="https://dev.to/webpadawan/the-journey-of-web-components-wrong-ways-lacking-parts-and-promising-paths-1d5a">The journey of Web Components: wrong ways, lacking parts and promising paths</a> - Serhii Kulykov</p> <p><a href="https://dev.to/richharris/why-i-don-t-use-web-components-2cia">Why I don't use web components?</a> - Rich Harris (svelte creator).</p> <p>I recommend you read the article but most importantly to read its comment. Especially <a href="https://gist.github.com/WebReflection/71aed0c811e2e88e3cd3c647213f0e6c">this gist</a> that is a direct answer to the previously mentioned article</p> <p><a href="https://dev.to/ben/why-the-react-community-is-missing-the-point-about-web-components-1ic3">Why the React community is missing the point about Web Components</a> - Ben Halpern.</p> <p>Once again the interesting part is mostly in the comments with a discussion between Dan Abramov et Benny Powers</p><![CDATA[Optimizing for high availability in time of crisis]]>/2020/04/high-availability-in-crisis//2020/04/high-availability-in-crisis/Thu, 09 Apr 2020 00:00:00 GMT<style> .boxes { display: flex; align-items: flex-start; } .boxes .box {} .boxes.row { flex-flow: row nowrap; } .boxes.row > * { width: 10%; flex-grow: 1; } .boxes.row > *:not(:first-child) { margin-left: 16px; } .boxes.column { flex-flow: column nowrap; } .box { padding: 16px 16px 16px 44px; width: 100%; border-radius: 3px; border-width: 1px; border-style: solid; border-color: transparent; background: rgba(235, 236, 237, 0.3) none repeat scroll 0% 0%; position: relative; font-size: 0.9em; margin: 8px 0; } .box.blue { background: rgba(221, 235, 241, 0.3) none repeat scroll 0% 0%; } .box a { font-weight: normal; } .box ol:last-child { margin-bottom: 0; } .box li { margin-bottom: 0; } .box aside { position: absolute; top: 16px; left: 12px; font-size: 1.2em; } </style> <nav class="box"> <aside>🚨</aside> <strong>In a rush? Here are the shortcuts:</strong> <ol> <li><a href="#increase-instance-size">Increase instance size</a></li> <li><a href="#use-autoscaling">Use autoscaling</a></li> <li><a href="#multiple-regions">Deploy to multiple regions (active/active sites)</a></li> <li><a href="#cache-static-assets">Cache static assets</a></li> <li><a href="#cache-db-queries">Cache long-running database queries and <li><a href="#store-sessions">Store sessions data properly</a></li> <li><a href="#rewrite-services">Rewrite bottleneck services</a></li> </ol> </nav> <p>These past few weeks have been tough for many websites. Some of them find themselves with an unprecedented increase in traffic (e.g. video conference software, online grocery stores, social media) and others with a dangerous reduction of users (e.g. car insurance, travel agencies, etc).</p> <p>Although both of those scenarios present severe risks for businesses, the second one often cannot be solved by technology alone. So the goal of this article is to help with the former and provide a set of quick actions to maintain a highly available website on AWS under unexpectedly high traffic.</p> <p>Most of the time in this situation, getting our website back online as soon as possible will be our top priority, which doesn't leave much time for writing or changing code (and would we want to add untested code into the situation anyway?). It also means that we're prepared to spend more on our infrastructure to accommodate the increased traffic (which may well pay for itself if that helps generate more business). For each recommendation I'll try to give a sense of the price increase involved.</p> <p>Some of the actions below are pretty generic (only assume a standard <a href="https://d0.awsstatic.com/whitepapers/aws-web-hosting-best-practices.pdf">2- or 3-tier web infrastructure</a>) and some are more context dependent. We'll cover the generic actions first and then explain in which context the specific actions apply.</p> <h1>Generic actions</h1> <h2 id="increase-instance-size">1. Increase instance size</h2> <div class="boxes row"> <div class="boxes column"> <div class="box"> <aside>⏳</aside> <strong>Time:</strong> &lt; 1h </div> <div class="box"> <aside>💵</aside> <strong>Money:</strong> from &times;2 to &times;100 on EC2 bill (you choose) </div> </div> <div class="box"> <aside>⛳️</aside> <strong>Effectiveness:</strong> increased capacity for cpu- or network-bound websites </div> </div> <p>This is the simplest and quickest action, although it can be a little bit expensive depending on the new instance type we choose.</p> <p>Keep in mind that increasing instance size does require replacing all the currently running ec2 instances (we can't increase the size of an existing instance), so this migration requires some planning. In particular, if our application is stateful, we risk losing data so our best option is probably to take snapshots of EBS volumes and recreate them for the new instances. In most cases however all that's needed is a redeployment of the app on the new instance(s).</p> <p>To save on ec2 costs remember to use <a href="https://aws.amazon.com/ec2/pricing/reserved-instances/">reserved instances</a> when possible.</p> <h2 id="use-autoscaling">2. Use autoscaling</h2> <div class="boxes row"> <div class="boxes column"> <div class="box"> <aside>⏳</aside> <strong>Time:</strong> 2h - 6h </div> <div class="box"> <aside>💵</aside> <strong>Money:</strong> from &times;0.1 to &times;10 on EC2 bill depending on autoscaling rules </div> </div> <div class="box"> <aside>⛳️</aside> <strong>Effectiveness:</strong> lifts all cpu- or network-bound blockers for stateless apps </div> </div> <p>This is a very good option if our app is already <a href="https://stackoverflow.com/a/2312980/2121761">stateless</a>.</p> <p>Autoscaling will automate the creation and termination of EC2 instances to match the fleet size to the current traffic. This will allow us to not over-provision instances to serve our users and save a lot of money. Remember that we can set a max number of instances in our fleet so we can stay in control of billing.</p> <p>If our app isn't stateless already, we may want to consider focusing development efforts to make it so in the short term. Although it's likely to require substantial change to the code, it's also going to unlock big performance improvements through parallelization (and autoscaling) so it might be worth the trade-off. If however we decide that making our application stateless isn't for us, there still are still things we can do to unlock performance, especially <a href="#store-sessions">option 6</a>.</p> <h2 id="multiple-regions">3. Deploy to multiple regions (active/active sites)</h2> <div class="boxes row"> <div class="boxes column"> <div class="box"> <aside>⏳</aside> <strong>Time:</strong> 0.5 day (with IAC, prepared) to 3 days (without IAC, unprepared) </div> <div class="box"> <aside>💵</aside> <strong>Money:</strong> &times;2 for the entire AWS bill </div> </div> <div class="box"> <aside>⛳️</aside> <strong>Effectiveness:</strong> doubled capacity </div> </div> <p>Another way to keep response times low worldwide and to massively increase our website's resilience is to deploy it in multiple regions, both serving traffic (also called <a href="https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/dns-failover-types.html#dns-failover-types-active-active">active/active deployment</a>). By this point we should already be using multiple availability zones in the first region and, by deploying a copy of the site to a second region, we will get:</p> <ul> <li>the capacity to serve double the normal number of users ;</li> <li>a site that's resilient against region-wide disasters.</li> </ul> <p>Infrastructure as code (or IAC, with e.g. <a href="https://www.terraform.io/">Terraform</a> or <a href="https://aws.amazon.com/cloudformation/">CloudFormation</a>) will make this deployment much easier and faster. If we don't use such a solution already, I wouldn't recommend setting one up just now, as it usually takes longer to prepare an IAC script than to provision an environment manually through the console. However we should definitely add this to our backlog for when things start to calm down.</p> <p>Once we've got our second region running, we'll need to split the traffic between the sites. This can be done with <a href="https://aws.amazon.com/route53/">Route53</a> which has several routing policy types to handle this, including latency-based or geolocation routing.</p> <p>Finally we probably need to use a "single" database for both regions. One way to achieve this is to create a read-replica of the existing RDS database in the second region (with async replication). Point all database access to the master database in the first region and in case of disaster in zone 1, we can promote the replica (AWS replaces the dns automatically, no code or configuration changes required).</p> <h1>Specific actions</h1> <h2 id="cache-static-assets">4. Cache static assets</h2> <div class="boxes row"> <div class="boxes column"> <div class="box"> <aside>⏳</aside> <strong>Time:</strong> 0.5 to 1 day </div> <div class="box"> <aside>💵</aside> <strong>Money:</strong> depends on asset sizes and volume downloaded, see calculator </div> </div> <div class="box"> <aside>⛳️</aside> <strong>Effectiveness:</strong> removes static files load on web servers </div> </div> <p>This is useful if web servers take time to serve each and every request for static assets (e.g. images, CSS and JS scripts, static html) or, even worse, re-generate them each time. In that case a great way to reduce the load on the servers is to have AWS serve the static assets for us.</p> <p>A standard solution to this problem is to store assets in S3 and serve them through AWS's CDN CloudFront. Using CloudFront enables caching of resources and uses edge locations around the world to ensure fast response everywhere, however it becomes expensive with many users. Use <a href="https://calculator.s3.amazonaws.com/">AWS's simple monthly calculator</a> (the newer <a href="https://calculator.aws/">AWS pricing calculator</a> doesn't support CloudFront yet) to estimate how much it would cost.</p> <h2 id="cache-db-queries">5. Cache long-running database queries and computed value</h2> <div class="boxes row"> <div class="boxes column"> <div class="box"> <aside>⏳</aside> <strong>Time:</strong> 1h to 1 day depending on web framework </div> <div class="box"> <aside>💵</aside> <strong>Money:</strong> 15 USD/month (with instance reservation) to 20k+ USD/month depending on requirements </div> </div> <div class="box"> <aside>⛳️</aside> <strong>Effectiveness:</strong> removes expensive computations from web servers </div> </div> <p>It's an almost unavoidable issue on dynamic websites with growing data: unoptimized database queries start taking longer, eventually blocking worker processes for several seconds at a time and seriously limit our ability to scale the number of concurrent users. While it's important to optimize database queries as much as possible, there's a simple way around this issue: caching query results.</p> <p>A typical way to implement application-level caching on AWS is <a href="https://aws.amazon.com/elasticache/">ElastiCache</a>. We'll pick between a <a href="https://redis.io/">Redis</a> and a <a href="https://www.memcached.org/">Memcached</a> backend based on what's easiest to implement in our application code and use the cached value as often as is reasonable.</p> <p>The same reasoning (and solution) applies for computed value that are not user-dependent and don't need to be recomputed on each request.</p> <h2 id="store-sessions">6. Store sessions data properly</h2> <div class="boxes row"> <div class="boxes column"> <div class="box"> <aside>⏳</aside> <strong>Time:</strong> 1h to 1 day depending on web framework </div> <div class="box"> <aside>💵</aside> <strong>Money:</strong> 10 USD/month (with instance reservation) to 1k+ USD/month depending on number of users </div> </div> <div class="box"> <aside>⛳️</aside> <strong>Effectiveness:</strong> removes lock acquiring bottleneck </div> </div> <p>In case our current application has to remain stateless and we're storing session information on the servers, make sure to store the session data in a way that allows concurrent access to the session store.</p> <p>For example, in some implementations (especially <a href="https://ma.ttias.be/php-session-locking-prevent-sessions-blocking-in-requests/">in PHP</a>), storing sessions in a single file can lead workers processes to waste a long time acquiring the session file lock <strong>on every request</strong>. An alternative to that is to store session data in memory, for example using <a href="https://aws.amazon.com/elasticache/">ElasticCache</a> on AWS (see action 5).</p> <h2 id="rewrite-services">7. Rewrite bottleneck services</h2> <div class="boxes row"> <div class="boxes column"> <div class="box"> <aside>⏳</aside> <strong>Time:</strong> depends on service </div> <div class="box"> <aside>💵</aside> <strong>Money:</strong> however much dev time worth of money </div> </div> <div class="box"> <aside>⛳️</aside> <strong>Effectiveness:</strong> removed application level bottleneck </div> </div> <p>This one is for last resort, when no other options are obviously available. Once we've identified (one of) the subsystems that are responsible for performance bottleneck(s) (for example using AWS X-Ray, New Relic or plain logs), it's time to consider whether a rewrite would yield significant improvements.</p> <p>In this sort of situation it's beneficial to start relying on cloud services to be able to build quickly and cleanly. AWS services like SQS (for decoupling), AWS Chatbot, Elastic Transcoder, Lambda, Amazon Polly and Kinesis for example are reliable building blocks to build solid, serverless workflows for chatbots, video transcoding, natural language processing or event streaming quickly. And there are many similar services we can use to build extremely scalable applications.</p> <hr> <div class="blue box"> <aside>☝️</aside> <strong>Need more help?</strong> I'd like to help you with your scalability issues! Contact me by <a href="mailto:nathang@theodo.co.uk">email</a> or on <a href="https://twitter.com/n4ng5l">twitter</a> and we'll try to find a solution that's right for you. 🙂 </div><![CDATA[How DUOLAB uses artificial intelligence to improve your skin]]>/2020/04/duolab-ai-skin-diagnosis//2020/04/duolab-ai-skin-diagnosis/Fri, 03 Apr 2020 00:00:00 GMT<p>For a long time people (including myself) have used the same skin care routine everyday. Only occasionally do we switch up our products for a different brand. The creams are opened and resealed multiple times a week, resulting in the products containing preservatives and other unnatural ingredients to prevent them spoiling. Innovative new start-up Duolab aims to improve the way we do skincare, their slogan being</p> <h2>‘Your life is no routine. Why would your skin care be?’</h2> <p>In November 2019 I started working for Theodo UK, a London-based software development agency which delivers bespoke web and mobile solutions for both start-ups and world-renowned clients alike. I was very excited to hear that my first project would be working within a scrum team to deliver DUOLAB’s web and mobile application to accompany the launch of their cutting-edge skincare product. Having a strong interest in skincare myself, I was satisfied with the project I was about to embark on!</p> <p>Everybody’s online presence and engagement is more important than ever given the current Coronavirus pandemic and home lockdown! DUOLAB is ahead of the curve by creating a future proof product to personalise your skin care without the need of a dermatologist or a shop assistant to provide a recommendation. You’re now able to improve your skin from the comfort of your own home.</p> <p>DUOLAB’s artificial intelligence (AI)-driven personalised skin care is revolutionising the way we care for our skin. Unlike traditional skincare treatments, DUOLAB utilises an AI skin diagnostic tool within its mobile app to calculate a tailored, personalised skincare protocol.</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/83081ec3fee2e1c2a28a834b8979d584/07f3a/skin_diagnosis.jpg" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 60%; position: relative; bottom: 0; left: 0; background-image: url('data:image/jpeg;base64,/9j/2wBDABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVGC8aGi9jQjhCY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2P/wgARCAAMABQDASIAAhEBAxEB/8QAGAAAAwEBAAAAAAAAAAAAAAAAAAQFAgP/xAAVAQEBAAAAAAAAAAAAAAAAAAAAAf/aAAwDAQACEAMQAAABt4UXSucw/8QAGxAAAgMAAwAAAAAAAAAAAAAAAQMAAhESFDH/2gAIAQEAAQUC9gyA6GutRnZvFW5L/8QAFBEBAAAAAAAAAAAAAAAAAAAAEP/aAAgBAwEBPwE//8QAFBEBAAAAAAAAAAAAAAAAAAAAEP/aAAgBAgEBPwE//8QAGBABAQEBAQAAAAAAAAAAAAAAAQAREEH/2gAIAQEABj8Cy3iGXkLf/8QAGhABAAIDAQAAAAAAAAAAAAAAAQARITFRYf/aAAgBAQABPyFROcosLiBYTHUK2Tw15Gr1vJ//2gAMAwEAAgADAAAAEJsf/8QAFhEBAQEAAAAAAAAAAAAAAAAAARAh/9oACAEDAQE/EF2f/8QAFREBAQAAAAAAAAAAAAAAAAAAARD/2gAIAQIBAT8QBn//xAAbEAEBAQEBAAMAAAAAAAAAAAABEQAhMUFR8P/aAAgBAQABPxCwAzvPjMiJ77cStH7wBxIaPTRod6f1wXA1QQ9d/9k='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="" title="Undertake a skin diagnosis on the Duolab website." src="/static/83081ec3fee2e1c2a28a834b8979d584/07f3a/skin_diagnosis.jpg" srcset="/static/83081ec3fee2e1c2a28a834b8979d584/d7fe6/skin_diagnosis.jpg 185w, /static/83081ec3fee2e1c2a28a834b8979d584/f4308/skin_diagnosis.jpg 370w, /static/83081ec3fee2e1c2a28a834b8979d584/07f3a/skin_diagnosis.jpg 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <p>The DUOLAB device blends a suitable combination of a moisturising base and a targeted concentrate which has been recommended to the customer by the skin diagnostic tool. The skin diagnostic process begins with a guided selfie which is inputted into the AI model. The AI assesses various skin traits, such as blemishes and wrinkles, which gives rise to more accurate recommendations than ever before. The selfie is followed by a short quiz which further determines the base-concentrate combination.</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/b849e8286725e3957aa9f6d9a8040d63/07f3a/duolab_capsules.jpg" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 72.43243243243242%; position: relative; bottom: 0; left: 0; background-image: url('data:image/jpeg;base64,/9j/2wBDABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVGC8aGi9jQjhCY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2P/wgARCAAOABQDASIAAhEBAxEB/8QAGAAAAwEBAAAAAAAAAAAAAAAAAAIDAQX/xAAVAQEBAAAAAAAAAAAAAAAAAAAAAf/aAAwDAQACEAMQAAAB7iUSKGlf/8QAGhABAAIDAQAAAAAAAAAAAAAAAQARAhAhMv/aAAgBAQABBQJannT0MW5//8QAFBEBAAAAAAAAAAAAAAAAAAAAEP/aAAgBAwEBPwE//8QAFBEBAAAAAAAAAAAAAAAAAAAAEP/aAAgBAgEBPwE//8QAGRAAAgMBAAAAAAAAAAAAAAAAARAAETFB/9oACAEBAAY/ApZL3q//xAAbEAEAAgIDAAAAAAAAAAAAAAABABFRcRAhMf/aAAgBAQABPyEhS1cpVlMVPSDotNQSXMaxx//aAAwDAQACAAMAAAAQ7w//xAAWEQADAAAAAAAAAAAAAAAAAAAQESH/2gAIAQMBAT8QiH//xAAWEQADAAAAAAAAAAAAAAAAAAAQESH/2gAIAQIBAT8QrH//xAAbEAEAAwADAQAAAAAAAAAAAAABABEhEDFBkf/aAAgBAQABPxAwhZEFiGclITBATpLJeOx6tja0wnmNfOP/2Q=='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="" title="Duolab base and targeted concentrates capsules" src="/static/b849e8286725e3957aa9f6d9a8040d63/07f3a/duolab_capsules.jpg" srcset="/static/b849e8286725e3957aa9f6d9a8040d63/d7fe6/duolab_capsules.jpg 185w, /static/b849e8286725e3957aa9f6d9a8040d63/f4308/duolab_capsules.jpg 370w, /static/b849e8286725e3957aa9f6d9a8040d63/07f3a/duolab_capsules.jpg 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <p>Your skin changes over time, so customers can retake the skin diagnostic when they choose - this will give them an up to date recommendation.</p> <p>Theodo UK’s comprehensive technology stack allows us to approach a broad range of clients with solutions to their business needs and problems. In the development of the DUOLAB mobile app, we used Facebook’s React Native framework to create a cross-platform, native app with an excellent user experience. Our use of Amazon web services (AWS) for hosting the mobile app and website was crucial in getting the project set up and running quickly. AWS gave us the reliability and flexibility that we needed to build our products with confidence.</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/e40830e68e86d3b880f15afb5ef92a44/07f3a/mobile_app.jpg" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 45.94594594594595%; position: relative; bottom: 0; left: 0; background-image: url('data:image/jpeg;base64,/9j/2wBDABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVGC8aGi9jQjhCY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2P/wgARCAAJABQDASIAAhEBAxEB/8QAGAAAAwEBAAAAAAAAAAAAAAAAAAIDAQT/xAAWAQEBAQAAAAAAAAAAAAAAAAAAAQL/2gAMAwEAAhADEAAAAemyMaRLj//EABoQAAICAwAAAAAAAAAAAAAAAAABAhEDIUH/2gAIAQEAAQUCx7k2idWjsT//xAAWEQEBAQAAAAAAAAAAAAAAAAAAARH/2gAIAQMBAT8BrX//xAAVEQEBAAAAAAAAAAAAAAAAAAABEP/aAAgBAgEBPwEn/8QAGhAAAgIDAAAAAAAAAAAAAAAAAAEQMRESIf/aAAgBAQAGPwJ7UUcwMUf/xAAcEAACAgIDAAAAAAAAAAAAAAAAEQEhMVFBYaH/2gAIAQEAAT8hkkttFqE6nQmDouDIz5GB/9oADAMBAAIAAwAAABCkP//EABcRAQADAAAAAAAAAAAAAAAAAAABETH/2gAIAQMBAT8QxaT/xAAVEQEBAAAAAAAAAAAAAAAAAAAAEf/aAAgBAgEBPxAj/8QAIBABAAIABQUAAAAAAAAAAAAAAQARITFBUaFhcYGRwf/aAAgBAQABPxBHxKtAeHfT3AJIuXVi9OkAmeHDV27TnTlPsze8/9k='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="" title="Duolab mobile application flow using React Native" src="/static/e40830e68e86d3b880f15afb5ef92a44/07f3a/mobile_app.jpg" srcset="/static/e40830e68e86d3b880f15afb5ef92a44/d7fe6/mobile_app.jpg 185w, /static/e40830e68e86d3b880f15afb5ef92a44/f4308/mobile_app.jpg 370w, /static/e40830e68e86d3b880f15afb5ef92a44/07f3a/mobile_app.jpg 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <p>Our best-in-class products and quick turnaround time are testament to our position in the global start-up studio, M33. Theodo’s M33 association enables us to leverage our relationships with other M33 companies to diversify our final product offerings. During the DUOLAB project, we worked alongside fellow M33 companies, Sicara and BAM Mobile Limited, who were responsible for the skin AI analysis and mobile app designs respectively. The shared philosophies of the seven M33 companies has enabled them to develop a robust, synergistic relationship which is capable of tackling the digital needs of modern companies.</p> <p>In January 2020, DUOLAB introduced their skincare solution to the world at CES® 2020 in Las Vegas - ‘The global stage for innovation’. The mobile app was demonstrated to consumers before its launch and it was well received.</p> <p>Since then DUOLAB has launched the product, mobile app and website within the UK. The products are on sale in selected L’Occitane stores and on the DUOLAB website. Given the current situation, DUOLAB has adapted to the needs of the consumer in a time when many other competitors haven’t. I hope that you can take these lessons from DUOLAB and stay ahead of the curve.</p><![CDATA[Serverless: a New Paradigm]]>/2020/04/serverless-a-new-paradigm//2020/04/serverless-a-new-paradigm/Fri, 03 Apr 2020 00:00:00 GMT<p>Since its creation in 2014, AWS' Serverless has delivered by reducing the cost of infrastructure management while increasing speed, agility, and scalability.</p> <h2>What is Serverless?</h2> <blockquote> <p>“Serverless allows you to build and run applications and services without thinking about servers” — AWS</p> </blockquote> <p>Serverless shifts more responsibility to Cloud Providers, allowing IT teams to build applications and services without managing the underlying architecture. Developers can send code, run databases, store data and manage security, without thinking in terms of servers, networks and patching. One classic example of a Serverless service is FaaS (Function as a Service), a Serverless approach to Compute that allows developers to send application code to their Cloud Provider and have it run without dealing with the underlying complexities of the server or network or other infrastructure components.</p> <p>Today the three main Cloud Providers for Serverless services are Amazon Web Services, Microsoft Azure, and Google Cloud.</p> <blockquote> <p>“A Serverless solution is one that costs you nothing to run if nobody is using it.” — Paul Johnston, ServerlessDays CoFounder</p> </blockquote> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <span class="gatsby-resp-image-background-image" style="padding-bottom: 19.459459459459456%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAECAYAAACOXx+WAAAACXBIWXMAABYlAAAWJQFJUiTwAAAA+klEQVQY0yWPvU7DQBCE83q8C89CgSigoIECCQkJKRJpEiJBgfhposQxcXCBwT9Egcs5sbF9d/44H9Ps7M7uzm6vLEuKomC73SGlpG0NLRaqQesapbWNGmNsvW1RSjmubZOW39RfqePG6G6KXn8w5vD4jKOTc676Q1Zr4YTV5SmD6z28jwe86YIwfMP3fRtDZ9BBBBfcDB/ZqS77N+wFk3vC+TNZFBC8DMgz3zUfPGWMxvvkn3dMZkv8uYcQwi2r6pp8I4l/Evx4TS631LbmLqRKIb2FZGQtp9BIJ/wa+5JuMFq5Bd1Ad0GHqqp4jyIWr0uyJHZciI3T/gD6ySmsINr9ugAAAABJRU5ErkJggg=='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="Cloud Providers Logos" title="Cloud Providers Logos" src="/static/b6bfac30bab95445a8bb07b363122eba/50383/cloud-providers-logos.png" srcset="/static/b6bfac30bab95445a8bb07b363122eba/1d79a/cloud-providers-logos.png 185w, /static/b6bfac30bab95445a8bb07b363122eba/1efb2/cloud-providers-logos.png 370w, /static/b6bfac30bab95445a8bb07b363122eba/50383/cloud-providers-logos.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </span></p> <h2>Prerequisites for Choosing Serverless in Your Organization</h2> <h3>Splitting Your Business Logic Into Events</h3> <p>The first step in architecting an event-driven system is to understand the business domain through its events. Domain-Driven-Design (DDD) is a great framework to achieve this and Event Storming workshops can help business and technical stakeholders understand the processes and systems at play in a given domain.</p> <p>These events are represented in business terms, e.g. Order Placed. But they ultimately trigger different Serverless services, for instance, running code, writing data, and authentication.</p> <h3>Code Compatibility</h3> <p>Because the feature is hosted and deployed on a Serverless Cloud Provider, it must follow its rules.</p> <p>As you shift more responsibility to your Cloud Provider, there are some constraints around the code you write. Primarily, the language teams use has to be compatible with their chosen Cloud Provider. For example, Google Cloud only accepts functions written in Go, Node.js, and Python. However, custom runtimes make any language possible.</p> <h2>Serverless Pros and Cons</h2> <h3>Pros:</h3> <ul> <li>Automatic scaling</li> </ul> <p>The Cloud Provider ensures that the function responds consistently every time it is called. For example, <a href="https://themeisle.com/blog/meta-description-examples/">iRobot faces a huge peak</a> at Christmas when new robot owners start their device, AWS will automatically scale compute power to absorb the load.</p> <p>In a Serverless paradigm, the Cloud Provider manages the running of your services, eliminating manual provisioning.</p> <ul> <li>You only pay for what you use</li> </ul> <p>Serverless Cloud Providers charge based on resource usage only as opposed to paying for allocated servers that most of the time are only used at a fraction of their capacity. No need to pay constantly for the extra buffer of compute power you might need during peak periods. And even better, if your function isn't used, you don't pay.</p> <p>When you consider that organizations use only a portion of the actual computational power of their servers, you realize the potential gain is significant. However, these servers that sit unused still consume energy, which leaves room for improvement to make the cloud computing industry greener.</p> <ul> <li>Reduce the Ops team size needed to run your systems, reducing your TCO (total cost of ownership).</li> </ul> <p>As there is less infrastructure managed by the team, they can focus on delivering value to users, rather than running the underlying systems.</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <span class="gatsby-resp-image-background-image" style="padding-bottom: 70.27027027027026%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAABYlAAAWJQFJUiTwAAABpUlEQVQ4y41UyW7CMBDN/39WL71VpeqlUgXCxNnsJs4GCVmm84Y6ClWoammwhZ/fMjYE4zjQ1pimSWoYBrper1KPxjzPCy6Y+QurDlR/WarqmpxzdLlcFvA4jgKc+FBbV1RXJdWMa5qGqqqisiwFgwKpEH48P1F2+KQ4Temw35Nl8vXw7navL/T+tqMkSUnriOIopuNR0fl8XsQDKONA07TUdZ0UAOKU1xhQBhh7dd1Q33eSAji0xccWwnmeyBpDEauFoaaTUqTDkBTPxtjFIQ4WeUGKHWmtKTyFUtbau/YEHphyXGxmWUYRxxmGcVGHQ6yxD7I4jgWH2bAZrOEemABWjcnE4Q1oyDAArnMWWhOWrmQxTYaJLbtPuZdJkkgvsX+LDCDfFDZRWZpJRAjdOxzJFYWIRlHEbdFCBgGYadv25nDd1PW8Hv5ScFERE8ZMiASZ9J5vO06Wp7MQbj1WT+4ju8ItDpU6Sq+VOonbru+3Hf4mvSN0N8I8z38edytR5fn4Z7Mm3Kp1ZMwgePQz/DPy7x6itlqyFhZCfPg/gkflCf+D+wailD8Xk2N6PwAAAABJRU5ErkJggg=='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="Testimonies" title="Testimonies" src="/static/25d8afcf1bd707642b512e647f0c5385/50383/testimonies.png" srcset="/static/25d8afcf1bd707642b512e647f0c5385/1d79a/testimonies.png 185w, /static/25d8afcf1bd707642b512e647f0c5385/1efb2/testimonies.png 370w, /static/25d8afcf1bd707642b512e647f0c5385/50383/testimonies.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </span></p> <div align="center">Source: <a href='http://serverless.com' target='_blank' noopener noreferer>Serverless.com</a></div> <h2>Cons:</h2> <ul> <li>Vendor Lock-In</li> </ul> <p>As you become serverless, your systems couple to the underlying Cloud Provider. This makes it difficult to move to another Cloud Provider later on, for instance, if their pricing changes. Certain large organizations are concerned by this dependency and therefore want to remain cloud agnostic. There are some solutions to reducing the extent of vendor lock-in, but these often come at the cost of some original benefits of Serverless.</p> <ul> <li>Requires a mindset shift (a new way of coding)</li> </ul> <p>Serverless can be massively empowering to developers, but it requires a mindset shift, correct coding, and consistent approaches to security and compliance. In this video, Nicole Yip, Senior Infrastructure Engineer at Lego, explains how their journey to Serverless was tightly linked to instilling the DevOps culture within the team.</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <span class="gatsby-resp-image-background-image" style="padding-bottom: 62.16216216216216%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAMCAYAAABiDJ37AAAACXBIWXMAABYlAAAWJQFJUiTwAAAB5ElEQVQoz51Sy07bUBC9sY3dOHFMEHnYzosmoRJSebQr2CDYlmxaAZUQCyBUgq666K4ItQg2BYk1v5BNXlIURYnySXEWsDzcuSTBBcKilo7u+M7MmXNnhsVjcTDGXkQ6lUYmnUEsGsNMZkb825YtfJLPB0WWR7FzuoaP0WmwhJMQF7Ik8yBpBB/zjS1EOPz2HaenfwZ5D7GqHnwk9ILICF5yOimZ/Avv53Fz9QvF3W0oisr9D3mmEQKz4hZC3Mjn8uJJiqw8I/YWoPskF/H35xf8OPgM20mN4o2gAUY98b/xi16lkimEJ8OYCk+JIi89VffrOD46xvnFJS6vrvF1cws2F0W+0FChwY3Z/Cyyb7NCJZ2O7UCdUKGpmgAVJZXk3/hUwMryMs7PzvD75ATR6YiH0LJEVZoiqSUiSkomkshlc0I59ZnsyCAxw/3rq2vY3dnBh8Wlf588HMqwT0/tp8Ois7BRQPGgiP29/cEKSc8JJUHySEYTHQfaSwK9wrtiZsgEe+c4iNAS8x5ZgSA0TeO9m3h1B4UawxC9pSGapomAHoCu62D1eh2lUgnVWg2VSgXlchmVahXVMaCYGo/tdrvodDpot9totVpoNptoNBpgrtuD2+uhx+G6Lvr9Pm7v7vC/3z0I/4DYccN1bgAAAABJRU5ErkJggg=='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="What is DevOps" title="What is DevOps" src="/static/322734a1687ece0ce4d50c1bfed8b9af/50383/what-is-devops.png" srcset="/static/322734a1687ece0ce4d50c1bfed8b9af/1d79a/what-is-devops.png 185w, /static/322734a1687ece0ce4d50c1bfed8b9af/1efb2/what-is-devops.png 370w, /static/322734a1687ece0ce4d50c1bfed8b9af/50383/what-is-devops.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </span></p> <ul> <li>Changes certain security practices</li> </ul> <p>Serverless removes certain security concerns, like patching operating systems as it is handled by the cloud provider who seamlessly applies security patches between two executions of a serverless function. Other security practices remain unchanged, like using a Web Application Firewall (WAF). But serverless also introduces an unprecedented level of flexibility in the definition of privileges at the function level. For instance you can specify that a given function should only be able to query certain elements of your database. This flexibility can prove itself dangerous as a rushed developer might be tempted to set lax security rules. This is why serverless requires adequate coaching and automation to ensure those rules are properly set.</p> <p>Find out more on security with our expert <a href="https://youtu.be/qhIzUHvllGw">Ben Ellerby’s talk</a> at ServerlessDays.</p> <h2>Cautions</h2> <ul> <li>If your code is used at high intensity, the Serverless solution might cost more for each query than the marginal cost your organization pays for conventional hosting.</li> </ul> <p>Parts of your application that would not benefit from permanently allocated resources can still be outsourced to a Serverless function. Authentication is a classic example where you would use a serverless authentication service like Cognito.</p> <h2>Use Cases</h2> <p>Serverless architectures have flourished. And use cases have grown in a broad range of industries and companies of all sizes such as:</p> <ul> <li>Netflix: Uses AWS for automatically encoding media files, automation of backup management, monitoring of other AWS services.</li> <li>PayPal: Has moved mission-critical workload to Google Cloud and transformed the way its 5,000 developers work.</li> <li>CodePen: A single SRE (Site Reliability Engineer) manages a Serverless infrastructure capable of responding to more than 200,000 requests in times of high demand, reducing the cost of managing infrastructure.</li> <li>Lego: After a system failure during Black Friday, the team decided to move to Serverless to handle peaks for their online shop. Their team now quietly watches their systems auto-scale.</li> </ul> <h2>In-House Serverless (An Alternative to Cloud Providers)</h2> <p>Since 2016, various frameworks have been gaining momentum, including OpenFaas and Knative. These solutions make it possible to no longer depend on Cloud Providers like AWS or Azure and allow you to replicate the code execution aspect of Serverless applications based on Kubernetes to host them yourself. However, your organization is again responsible for the servers on which your code is run.</p> <p>Large corporations usually have large IT departments, maybe a few data centers and often an intricate plan to migrate to the cloud. Such companies could benefit from the speed and agility provided by Serverless Compute (Functions as a Service or FaaS) that would be enabled deploying Knative on their own infrastructure. The complexity of managing would not be outsourced to a Cloud Provider but handled by the existing IT team but the development team would highly benefit from the possibilities that Serverless has to offer.</p> <hr> <h2>Conclusion</h2> <p>Serverless is recognized for bringing more agility and scalability to the organization. In addition, Serverless can help large organizations reduce the time-to-market of new products. </p> <p>By breaking up a monolithic architecture in microservices, developers can focus on business-specific code and deliver customer-centric solutions. Where appropriate, it is an excellent solution to reduce the cost of managing the infrastructure and increasing the impact of your developers on your business.</p> <p>Whether you want to start a new project with state-of-the-art technology or are looking to reduce the cost of managing your infrastructure, Serverless is a technology to seriously consider.</p><![CDATA[Ensure code quality; create your own ESLint rules]]>/2020/04/create-your-own-eslint-rules//2020/04/create-your-own-eslint-rules/Wed, 01 Apr 2020 00:00:00 GMT<h1>Ensure code quality; create your own ESLint rules</h1> <h2>Introduction</h2> <p>We used ESLint rules to enhance code quality. This article is about how one would proceed to create one's own custom rule. We used this procedure at <a href="https://www.theodo.fr">Theodo</a> to make sure our frontend code was XSS free on every project. </p> <p>If you are not familiar with XSS vulnerabilities <a href="%5Bhttps://owasp.org/www-project-cheat-sheets/cheatsheets/Cross_Site_Scripting_Prevention_Cheat_Sheet%5D(https://owasp.org/www-project-cheat-sheets/cheatsheets/Cross_Site_Scripting_Prevention_Cheat_Sheet)">here is an article</a> from OWASP that summarizes well what it is about. </p> <h2>Context and Needs</h2> <p>There are over 80 developers at Theodo, working on over 30 different projects. This setup presents some challenges that many tech companies face: </p> <ul> <li>Code consistency and maintainability. </li> <li>Continuous improvement of the code quality</li> <li>Optimization of the development environment</li> <li>A straightforward onboarding for the newcomers on the different projects</li> </ul> <p>We came up with a solution that is easy to implement and extremely effective: creating a custom linting rule for ESLint.</p> <p>I will use an example of a rule we needed for our frontend projects with an objective of no know XSS vulnerability in our code. </p> <p>We started by implementing the rule to React and Vue, which covers the majority of our projects. There are a few differences to take into account when developing a rule for each of these frameworks. We will look at them in this article. </p> <p>If you encounter issues configuring ESLint, especially alongside other fromatting tools such as Prettier, I recommend <a href="https://blog.theodo.com/2019/08/empower-your-dev-environment-with-eslint-prettier-and-editorconfig-with-no-conflicts/">this article</a> that tremendously helped me to configure my development environment.</p> <h2>Strategy</h2> <p>Nobody is infallible. Creating an automated check that alerts the developer is more reliable than documentations on the project or code reviews. Don't get me wrong, code reviews are essential, as is a well-written documentation, but it is often not enough to enforce a coding practice. Having an automated rule also means shorter code reviews and an easier time following the best practices.</p> <p>Implementing a rule that will be in the IDE also allows you to suggest the best way to do something, as soon as it is written. This is a roundabout way to train your developers, that isn't time-consuming for you.</p> <h2>How to implement an ESLint rule</h2> <p>To understand how a rule is coded, we need to take a look at our code's Abstract Syntax Tree and how we can use it.</p> <h3>The Abstract Syntax Tree (AST)</h3> <p>An AST is a graph representation of your code. ESLint uses it to navigate your code and to decide which actions to take and where. </p> <p>The screenshot below is an example of a simple Hello World! AST of a React script, containing a dangerouslySetInnerHtml. We can see the tree structure and the names of the different nodes. For example, the <code>FunctionDeclaration</code> node contains the information from the <code>function</code> keyword, up to the closing tag. </p> <p>The screenshot below is an example of a simple Hello World! AST. We can see the tree structure and the different nodes' names. For example, the <code>FunctionDeclaration</code> node contains the information from the <code>function</code> keyword, up to the closing tag. </p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/fad2d5968b00c141964443597e925374/50383/ast_example.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 77.83783783783782%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAQCAIAAACZeshMAAAACXBIWXMAAAsSAAALEgHS3X78AAABiElEQVQoz5WTbW/TMBCA8++p2NKmGlI1iYn9IeAzMDbWZo4Tn+/81sRumhknZTAmsZbTo/PbPfKL5MyAZl9R8r6zbmft0HVD2+7dNvpAiB8/ff52c7Mpy6qqOOesGlvGmBACEbNhGJw1IAVICQB2G7Z+nyZjjCHsqpqta8YB0pocCw5AMoko853v210f9kPfx/g4DCNxCu+DaEoueCpUStGU1SFNkXWu6yi00Hrr4t/hQxD1RlT3VTrWuJN6QRb/HT7sBP/B2Zfv9QOX3KgGqaFnTPLjEy9k3yPecVit5WVJl0KvjFvZZ7y6s++JblHOKsyVmSubG5tblxs35sQJMp4xyDktjSmUKbQpjF1YN3JUvtNyVsu3YArnfjm/OS6TnK2b+a28aPTC/a9sTd7gGdnCTIJ74rRj6/NNsyjhnaAl6UJNpE66+QmyXXIsGF6AWh4eTE2cJBO9UTQjPbfu3G3/kIZHZMR7ie8lXpG+1ubDAWMT14nXZS+krEAyAQ2k7gSmLwVEMvET4+h2jLnE0jQAAAAASUVORK5CYII='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="ast-example" title="ast-example" src="/static/fad2d5968b00c141964443597e925374/50383/ast_example.png" srcset="/static/fad2d5968b00c141964443597e925374/1d79a/ast_example.png 185w, /static/fad2d5968b00c141964443597e925374/1efb2/ast_example.png 370w, /static/fad2d5968b00c141964443597e925374/50383/ast_example.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <p>This screenshot is taken from <a href="http://astexplorer.net">astexplorer.net</a> which is an extremely useful resource to visualize and navigate your AST. </p> <p>It works nicely for Javascript and React. However, when exploring your Vue AST, you will need to find another way for the <code>&#x3C;template></code> part of a file, which is parsed as HTML. AST explorer's own HTML configuration does not match perfectly with what my Vue code outputs. I used <a href="%5Bhttps://ktsn.github.io/vue-ast-explorer/%5D(https://ktsn.github.io/vue-ast-explorer/)">this tool</a> instead, which is the same, except only for Vue.js</p> <p>We'll come back to the particularities of Vue.js later in this article.</p> <h3>How ESLint uses your code's AST</h3> <p>ESLint uses a visitor that will parse your AST for each file it is allowed in. In this visitor function, you can define actions for each node type. If we use the same example as above, we could write something like this.</p> <div class="gatsby-highlight" data-language="js"><pre class="language-js"><code class="language-js"> <span class="token keyword">const</span> <span class="token function-variable function">create</span> <span class="token operator">=</span> <span class="token parameter">context</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">return</span> <span class="token punctuation">{</span> <span class="token function">Program</span><span class="token punctuation">(</span><span class="token parameter">node</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">try</span> <span class="token punctuation">{</span> console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span><span class="token string">'Entering the program!'</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token keyword">catch</span> <span class="token punctuation">(</span>error<span class="token punctuation">)</span> <span class="token punctuation">{</span> context<span class="token punctuation">.</span><span class="token function">report</span><span class="token punctuation">(</span>node<span class="token punctuation">,</span> error<span class="token punctuation">.</span>stack<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token function">FunctionDeclaration</span><span class="token punctuation">(</span><span class="token parameter">node</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">try</span> <span class="token punctuation">{</span> console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span><span class="token template-string"><span class="token template-punctuation string">`</span><span class="token string">Function </span><span class="token interpolation"><span class="token interpolation-punctuation punctuation">${</span>node<span class="token punctuation">.</span>id<span class="token punctuation">.</span>name<span class="token interpolation-punctuation punctuation">}</span></span><span class="token string"> is defined</span><span class="token template-punctuation string">`</span></span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token keyword">catch</span> <span class="token punctuation">(</span>error<span class="token punctuation">)</span> <span class="token punctuation">{</span> context<span class="token punctuation">.</span><span class="token function">report</span><span class="token punctuation">(</span>node<span class="token punctuation">,</span> error<span class="token punctuation">.</span>stack<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span></code></pre></div> <p> In the example, I use the root node <code>Program</code> to trigger some action, and the <code>FunctionDeclaration</code> node. </p> <p>This is a very basic example, where we can already see a few important things about the way ESLint will work.</p> <ol> <li> <p>The context object</p> <p>The context passed as an argument here contains information relevant to the rule. We won't use it here, but if you want more info here's the official documentation: <a href="https://eslint.org/docs/developer-guide/working-with-rules">https://eslint.org/docs/developer-guide/working-with-rules</a></p> </li> <li> <p>ESLint uses the node names to trigger some actions. Here, a <code>console.log</code>. These actions will be triggered for <em>each node</em> with the same name in the AST. For example, there is only one Program, but usually more than one function declaration. In the above example, our console output will look like:</p> <div class="gatsby-highlight" data-language="bash"><pre class="language-bash"><code class="language-bash">Entering the program<span class="token operator">!</span> Function example is defined Function helloWorld is defined</code></pre></div> </li> <li> <p>Each node has access to its sub-nodes, and also to its parent node if it has one. This is why we can use <code>node.id.name</code> to get the name of the declared function. The <code>Identifier</code>node <code>'id'</code> is a sub-node of the <code>FunctionDeclaration</code>node, just like the <code>BlockStatement</code> node named <code>'body'</code>.</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 514px; " > <a class="gatsby-resp-image-link" href="/static/b1f656b47c0ff03b587a54eac3396bb9/dea13/ast_function.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 67.56756756756756%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAIAAACgpqunAAAACXBIWXMAAAsSAAALEgHS3X78AAABRElEQVQoz5VS7U7DMAzs+78a0iRgHd3GYGs+bF/sJIMXIG3Hxx/YuFitE8n25S7dxqfBxdVRHhzundydZE9CimjGquVPdOcMOa1y8huJB9BO4hPHZ/DAdABfKTYDwsAIB0gt5b3Wt88413qlWJU07n1KrOlci+UZpeRyHR1SSPTCSI6Y2oY5qbaJ+bvLpc/y+zxeihESj9tYBq+7qKMYMwvAImh0RCQBCSTS8lKLpNZc6+XOSuIe4QelQ1DtKbBpLTehy8rs1vCbKUA90+us3I3F0iabuD1k4JjLPzBNhn9C2JqpmbWjr7G1XvGqE4zwPXHsnW6DDsHWTo+sk0DAz2hCzbKhlTUt27aD0jSZx4DcfDpF8s03y7Ywscn4+ZNtSmxhtxg2PRIeH8T1Ct/MaWt2NF9oz8Tr788zZSVLPCX/kquUD5d0BTDHiS9rAAAAAElFTkSuQmCC'); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="ast-function" title="ast-function" src="/static/b1f656b47c0ff03b587a54eac3396bb9/dea13/ast_function.png" srcset="/static/b1f656b47c0ff03b587a54eac3396bb9/1d79a/ast_function.png 185w, /static/b1f656b47c0ff03b587a54eac3396bb9/1efb2/ast_function.png 370w, /static/b1f656b47c0ff03b587a54eac3396bb9/dea13/ast_function.png 514w" sizes="(max-width: 514px) 100vw, 514px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> </li> <li>Our approach to implementing our custom rule</li> </ol> <p><strong>Test-Driven Development</strong> works like a charm in this particular case. We obviously want to test our rule, but there are other advantages to doing so. </p> <p>For those who are not familiar with Test Driven Development (TDD), it consists of three phases.</p> <ol> <li>Write a test case you want your code to pass. Run the tests. It fails. (Red test)</li> <li>Write code to pass your test, <em>not more.</em> Run the tests, it should pass. (Green test)</li> <li><strong>Refactor</strong> your code.</li> </ol> <p>This process is often shorthanded in Red-Green-Refactor.</p> <p>The TDD method is awesome and you should use it where it fits. It does fit here for a few reasons. </p> <ol> <li>One of them is that it allows us to ship a stable and well-documented version, with examples based on the tests. </li> <li>In the context of our project, we were able to quickly test our custom ESLint rules in real-life projects, and obtain feedback. It was easier to convert the feedback in tests first (red tests), and then write some code to fix the issue (green tests). </li> <li>In our case, many of our tests were easy to write, since they often were chunks of code we wanted to be flagged, or not. However, turning them green is less trivial. Here's an example:</li> </ol> <div class="gatsby-highlight" data-language="js"><pre class="language-js"><code class="language-js"> <span class="token comment">// Test cases we do not want to catch with our rule</span> <span class="token function">testCase</span><span class="token punctuation">(</span><span class="token template-string"><span class="token template-punctuation string">`</span><span class="token string"> &lt;template> &lt;div class="content"> &lt;div v-html="message" /> &lt;/div> &lt;/template> &lt;script> import DOMPurify from 'dompurify'; const rawHtmlInput = '&lt;a onmouseover=\"alert(document.cookie)\">Hover me!&lt;/a>'; export default { name: 'HelloWorld', data () { return { message: DOMPurify.sanitize(rawHtmlInput) } } } &lt;/script> </span><span class="token template-punctuation string">`</span></span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token comment">// Test cases we do want to catch</span> <span class="token function">testCase</span><span class="token punctuation">(</span><span class="token template-string"><span class="token template-punctuation string">`</span><span class="token string"> &lt;template> &lt;div class="content"> &lt;div v-html="message" /> &lt;/div> &lt;/template> &lt;script> import DOMPurify from 'dompurify'; const rawHtmlInput = '&lt;a onmouseover=\"alert(document.cookie)\">Hover me!&lt;/a>'; export default { name: 'HelloWorld', data () { return { message: rawHtmlInput } } } &lt;/script> </span><span class="token template-punctuation string">`</span></span><span class="token punctuation">)</span><span class="token punctuation">,</span></code></pre></div> <p>These two cases are similar, except in the <code>message</code> part, one is sanitized by DOMPurify whereas the other isn't. </p> <p>And more generally, tests protect us from breaking previously working code, and catch some edge cases during the development phase. </p> <ul> <li>Managing the Vue AST</li> </ul> <p>As teased above, let's look at the particularities of Vue files when writing an ESLint rule.</p> <p>Vue files are split in 3 parts, a <code>&#x3C;template></code>part, a <code>&#x3C;script></code>part and a <code>&#x3C;style></code>.</p> <p>The AST, in this case, does not include a unified version of the file. It is split in the same way. This means our visitor function will have to be split as well. </p> <p>I use a standard ESLint parser for Vue, that exposes a function which takes the context and 2 visitors as arguments. One for the template, and one for the script.</p> <p>Here is a code snippet to give you a better idea:</p> <div class="gatsby-highlight" data-language="js"><pre class="language-js"><code class="language-js"> <span class="token keyword">const</span> <span class="token function-variable function">create</span> <span class="token operator">=</span> <span class="token parameter">context</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token comment">// The script visitor is called first. Then the template visitor</span> <span class="token keyword">return</span> utils<span class="token punctuation">.</span><span class="token function">defineTemplateBodyVisitor</span><span class="token punctuation">(</span> context<span class="token punctuation">,</span> <span class="token comment">// Event handlers for &lt;template></span> <span class="token punctuation">{</span> <span class="token function">VAttribute</span><span class="token punctuation">(</span><span class="token parameter">node</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">try</span> <span class="token punctuation">{</span> <span class="token comment">// SOME CODE HERE</span> <span class="token punctuation">}</span> <span class="token keyword">catch</span> <span class="token punctuation">(</span>error<span class="token punctuation">)</span> <span class="token punctuation">{</span> context<span class="token punctuation">.</span><span class="token function">report</span><span class="token punctuation">(</span>node<span class="token punctuation">,</span> <span class="token template-string"><span class="token template-punctuation string">`</span><span class="token interpolation"><span class="token interpolation-punctuation punctuation">${</span>utils<span class="token punctuation">.</span><span class="token constant">ERROR_MESSAGE</span><span class="token interpolation-punctuation punctuation">}</span></span><span class="token string"> \n </span><span class="token interpolation"><span class="token interpolation-punctuation punctuation">${</span>error<span class="token punctuation">.</span>stack<span class="token interpolation-punctuation punctuation">}</span></span><span class="token template-punctuation string">`</span></span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token comment">// Event handlers for &lt;script> or scripts</span> <span class="token punctuation">{</span> <span class="token function">Program</span><span class="token punctuation">(</span><span class="token parameter">node</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">try</span> <span class="token punctuation">{</span> <span class="token comment">// SOME OTHER CODE THERE</span> <span class="token punctuation">}</span> <span class="token keyword">catch</span> <span class="token punctuation">(</span>error<span class="token punctuation">)</span> <span class="token punctuation">{</span> context<span class="token punctuation">.</span><span class="token function">report</span><span class="token punctuation">(</span>node<span class="token punctuation">,</span> <span class="token template-string"><span class="token template-punctuation string">`</span><span class="token interpolation"><span class="token interpolation-punctuation punctuation">${</span>utils<span class="token punctuation">.</span><span class="token constant">ERROR_MESSAGE</span><span class="token interpolation-punctuation punctuation">}</span></span><span class="token string"> \n </span><span class="token interpolation"><span class="token interpolation-punctuation punctuation">${</span>error<span class="token punctuation">.</span>stack<span class="token interpolation-punctuation punctuation">}</span></span><span class="token template-punctuation string">`</span></span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">;</span></code></pre></div> <p>From there it is basically the same approach as a single visitor function, like with vanilla Javascript, duplicated in each function.</p> <p>I used the library eslint-vuejs (<a href="https://eslint.vuejs.org/developer-guide/#working-with-rules">https://eslint.vuejs.org/developer-guide/#working-with-rules</a>) that already provides a function to manage the template and script part at the same time : </p> <p><code>context.parserServices.defineTemplateBodyVisitor(visitor, scriptVisitor)</code> where both <code>visitor</code> and <code>scriptVisitor</code> are functions like the one presented before.</p> <p>And there you go, you can check any number of things you want to alert your user about.</p> <p>In our Vue example, we checked for <code>v-html</code> in the template and checked if it was verified with libraries like DOMPurify. We know <code>v-html</code> is generally not the way to go, but sometimes it is a necessity, and we wanted to make sure we weren't introducing a potential XSS. </p> <p>Here is what the custom rule looks like in action:</p> <p><img src="/78b0be7da5e90159eb975956c6419235/risxss-demo.gif" alt="risxss-demo.gif"></p> <p>If you also think that integrating a custom ESLint rule to produce better code is awesome, definitely check out the project I used as an example, it open-sourced here: <a href="https://github.com/theodo/RisXSS">https://github.com/theodo/RisXSS</a></p> <h2>Conclusion</h2> <p>In conclusion, integrating custom ESLint rules in your projects will help improve the code quality, the lead time of your features, since fewer mistakes will make it to the code review part of the process, and the developer's experience. </p> <p>With a rule matching a bad practice, you can easily suggest a good practice right in the IDE, and the developers will quickly and painlessly learn it. </p> <p>In our case, after some pilots, the rule was used in the development environment of the developers and guarantees us a certain level of code security. </p> <p>If you want to dive deeper into the custom rules world of ESLint, a good place to start is the official documentation for developers: <a href="https://eslint.org/docs/developer-guide/working-with-rules">https://eslint.org/docs/developer-guide/working-with-rules</a> </p> <p>I will also link here two articles that helped me greatly with ESLint custom rules. If you found mine interesting, you should definitely check these out.</p> <p><a href="https://flexport.engineering/writing-custom-lint-rules-for-your-picky-developers-67732afa1803">Writing Custom Lint Rules for Your Picky Developers</a></p> <p><a href="https://medium.com/neighborhoods-com-engineering-blog/custom-eslint-rules-for-faster-refactoring-2095e69bde08">Custom ESLint Rules For Faster Refactoring</a></p><![CDATA[How real developers use a CMS to build a showcase website with Netlify, Nuxtjs, and Contentful]]>/2020/03/content-driven-static-website//2020/03/content-driven-static-website/Tue, 24 Mar 2020 00:00:00 GMT<h1>Why build a CMS driven showcase website?</h1> <p>Here you are, you just started a project to build the latest shiny showcase website for your client. Now comes the difficult part: you want to offer your client the best bang for his bucks, not wanting to redo part of the website each time the content changes. In the same way, you are probably not in charge of coming up with the shiny design but coding it, why would you lose time adjusting wordings, coming up with translations?</p> <p>That's where a Content Management system (CMS) becomes handy, giving your client an interface to edit the content of his site without wasting its precious developers' time. Now you might be thinking 'A CMS, last time I gave a go-to WordPress, I wasn't convinced !' and I agree with you. Today CMS fall into 2 usages:</p> <ul> <li>A what you see is what you get, that generates a website: WordPress, Drupal are some examples</li> <li>headless CMS which are a back-end only content management system built as a content repository that makes content accessible via a RESTful API: WordPress, Drupal are part of the game here as well, but we will use in this tutorial a concurrent specialized for this usage: <a href="https://www.contentful.com/">Contentful</a> to convince you that working with a CMS can be enjoyable.</li> </ul> <h1>Why use a server-rendered static website for a showcase website?</h1> <p>Doing so leverages the advantages of using a progressive web application with server-side rendering (<a href="https://blog.theodo.com/2018/04/react-server-side-rendering/">more about SSR here</a>) without the cost of running a server since you just need to serve your files. Furthermore, we leverage the following advantages for a showcase website:</p> <ul> <li>SEO friendly</li> <li>Performance</li> <li>Monthly infrastructure cost</li> </ul> <h1>Why use Netlify &#x26; Nuxtjs</h1> <h2>Netlify</h2> <p>Infrastructure wise our requirements are very small, we only need a CDN to distribute our site. We could use any major cloud provider directly, however, I decided to go with <a href="https://www.netlify.com/">Netlify</a> service for the following reasons:</p> <ul> <li>The integration is exceptional: creating a CI that builds and deploys your website in production has never been easier.</li> <li>Https works out of the box as long as you have your own domain.</li> <li>Netlify service is built on top of AWS which means we don't have to worry about reliability.</li> </ul> <h2>Nuxtjs</h2> <p><a href="https://nuxtjs.org/">Nuxtjs</a> is a server-side rendering framework built on top of Vuejs. For our case, Nuxtjs allows for dead easy creation of a server-side rendered static website.</p> <h1>Building a CMS driven showcase website</h1> <img alt="Create a Contentful CMS" src="/a3461bbd9853033d746193743c6b65f9/contentful-logo.svg" style="margin-top:60px;width:90%;"> <h2>Creating a Contentful CMS</h2> <p>First, as the website is content-driven, we will start by creating the CMS using Contentful to provide content to the website.</p> <p>Start by creating an account on <em><a href="https://www.contentful.com/sign-up/">https://www.contentful.com/sign-up/</a></em> (use free trial for this tutorial) Once you sign up, you can create a new space (which is Contentful way of creating a project)</p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/97df91696c4d26537e9b2e3f7e427203/50383/home_contentful.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 50.27027027027027%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAKCAIAAAA7N+mxAAAACXBIWXMAAC4jAAAuIwF4pT92AAABoElEQVQoz02PSW8TQRCF+z9yM0FAAocQIAgkRCDiz3DPEa5JBMhCAiVSNCzCZjyexZ4N997VyzQ9Hsu49alar1WvXhcaHTy+9eD45M7D/aPXo6en95/13Dt+s9/X04Pnb+8+Odk7ehW4/ejl6PA/e4cvUMYh4yqQCyiknrY4ypc/l02ULb9NZlfx/HtRXsfpzbyYYRYaQtvQmawoGkQmNiOKINc18LtspvXfOeG1NhXolIqhZ0hKMENb3SOgJqQqy6quwqGUtivsvT/7cHE+vgLvk61fQIopyhj0DPMkpA29/rW4mSyiaTkrGSilvX/3/uPF12hrTsPXJFRcoIzCQEqhYBC3+vNUfYlh/Ed+mnAKFqxrTCc6j8EkTIalSgncOqEUKojJid5AzWLF8xozzjpnvLfaOu26rnNE6ZUy0jrTy7CKN8agnJhiTY5NyWyUy8sfhAgtlOFKK221c877GreUE79zwjMiyq7DzVAxV5TzllAW3M4bG3Cuz+o2ibvmcEndNdwuaR8OxkWVqKUUYMcJrSQNjvMJvoz72F17MP8DACYVuEtl3PkAAAAASUVORK5CYII='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="Create a new project in Contentful step 1/2" title="Create a new project in Contentful step 1/2" src="/static/97df91696c4d26537e9b2e3f7e427203/50383/home_contentful.png" srcset="/static/97df91696c4d26537e9b2e3f7e427203/1d79a/home_contentful.png 185w, /static/97df91696c4d26537e9b2e3f7e427203/1efb2/home_contentful.png 370w, /static/97df91696c4d26537e9b2e3f7e427203/50383/home_contentful.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/491abceaa2290983c8080b2a82c2a711/50383/contentful_new_space.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 50.27027027027027%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAKCAIAAAA7N+mxAAAACXBIWXMAAC4jAAAuIwF4pT92AAABaklEQVQoz22PO0/DMBCA+0OYaBEDLRKirdRHkqZJ7DixEzdO8tNYmGBgY6cSCJD4FSxFtIjXAILSF0M5J1FRA9Kn09l3351duL0bDh4eP+5HxweHG9t7W7v1UqVWqtSLlVoxTcq1zXJVspOnMHp6GT2/vg2Gp0cniu3rbqARXyNctX0FMQV7kENMS10qVsCxMJ5M3z/Hy+Xy7PqmhZjBQigYCR3CdafXdQOThel9Xp4tviezuZTPL9uIISYsl5uAwxENLNojPK4qVkMnmMdry0Gezhdf05mUL65ANqlQka9irmGuE4GZsHm8r+Km4WI/0t3MhIcY9I9s0KiNRSsjQCzEXmh5EfZii4V6Zgq4hLm/cl/KngU1p4dYQPwQsD0gAmCKJQfJnPgSKK1tVpDXccImyhbiJMpuHps0nRinZl7uJzKWS4RFwRc2dCRAq8PDlfa/DM+G2ZojzORX8GeTBTaPGiho4MDlefkHRT/ywBCCCJQAAAAASUVORK5CYII='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="Create a new project in Contentful step 2/2" title="Create a new project in Contentful step 2/2" src="/static/491abceaa2290983c8080b2a82c2a711/50383/contentful_new_space.png" srcset="/static/491abceaa2290983c8080b2a82c2a711/1d79a/contentful_new_space.png 185w, /static/491abceaa2290983c8080b2a82c2a711/1efb2/contentful_new_space.png 370w, /static/491abceaa2290983c8080b2a82c2a711/50383/contentful_new_space.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <p>In the popup dialog set the type of space (you can upgrade later), the space name (the project name):</p> <ul> <li>Space type: <code>free</code></li> <li>Space details: <code>data_driven_static</code>, <code>empty space</code></li> </ul> <p>You have successfully created a new contentful project <img alt="party popper emoji" src="/eceeb77e2efaae09977e9ca92a0464cb/party_popper_emoji.svg" style="max-height: 20px;"></p> <h2>Creating the first content</h2> <p>Contentful is organized between:</p> <ul> <li><code>Content Models</code> which is where you define the structure of the content</li> <li><code>Content</code> which is where you can contribute (create content)</li> <li><code>Settings</code> where you will find some useful settings later on.</li> </ul> <h3>Create the content model</h3> <p>To create our first content we will use the Content model tab, click on <code>add content type</code></p> <p>When creating a new content type, we have to fill in some important information:</p> <ul> <li>The Name of the content type, which is the displayed name for the content type: we will use <em>Section</em>.</li> <li>The API Identifier is the name of the entity when querying the Contentful API. In most cases it’s better to keep the given API Identifier, to stick as close as possible to the content type name to ease identifying the content types in the code.</li> <li>Lastly, I recommend filling in the description of the content type, I will use: <em>Basic block of content for a page</em>.</li> </ul> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/6d5bc1584feedc397112803422aa35e4/50383/create_content_type.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 50.27027027027027%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAKCAIAAAA7N+mxAAAACXBIWXMAAC4jAAAuIwF4pT92AAABfElEQVQoz2WRy07CQBSGeREEWSjoQiC6IQJK6YV2CnSmtaABE/V5fAw2JiyMBi8EBYkPYdQoghdE3QmElnpoBQUmXyadM+fr+dPaHPPeGRPXwrJrEVhxun1wtOrWbp9bgn3W4wecHr/DDfjsHq+NEQmLSCyhAFxc4RIKwum4vAkklK0RIkkPe2QOlDihRGzjzdKQjSDF5w7yj/Xnm7v7Wr0BPNSeWp9f+cOjMIOQpFICCbE4zOF1ntj+mQqfVANrbLFUNgxD13XDXJqmwV6pXgejPMIqOAFaAoLsuCxIA7lwWoTuXk/rmwse4HhRqYaiAkxmRcIgzIky5B+TEU5B7MLJeV83Op2OpulAt9v7k7FKI0LxEoNkVpySVyOxUvnKTDsZOwSxB5NlGmEGwfyp2BQt7udzl2+3L43XVusDaDbf29/t48JZmEbQADNpgcArJmXrT+BURs3upDK7qewv6e09OZ2FKz6pgBmJSVEB09Oy5cNXmQCK1i3MHPEDdxcEJxqocpEAAAAASUVORK5CYII='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="Create a new content type in contentful" title="Create a new content type in contentful" src="/static/6d5bc1584feedc397112803422aa35e4/50383/create_content_type.png" srcset="/static/6d5bc1584feedc397112803422aa35e4/1d79a/create_content_type.png 185w, /static/6d5bc1584feedc397112803422aa35e4/1efb2/create_content_type.png 370w, /static/6d5bc1584feedc397112803422aa35e4/50383/create_content_type.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <p>Finally, confirm to create the new content type.</p> <p>Now, we have a new empty content type. To fill it, we will add some fields:</p> <ul> <li>A Title for the section</li> <li>A Text for a paragraph</li> </ul> <p>To do so, first, click on <em>add field</em>:</p> <ul> <li>For the type of the field, select: Text</li> <li> <p>For the settings of the Text field:</p> <ul> <li>Name: Title</li> <li>Short text</li> </ul> </li> <li>Click create and configure</li> <li> <p>Then in settings:</p> <ul> <li>Select: <em>This field represents the Entry title</em></li> </ul> </li> <li> <p>Switch to validation:</p> <ul> <li>Make it required</li> </ul> </li> <li> <p>Finally switch to the Appearance tab:</p> <ul> <li>Leave it as a single line</li> <li>Enter a Help text: <em>The section title</em></li> </ul> </li> <li>Click Save</li> </ul> <p>You successfully created the first field, we will create a second one for the paragraph:</p> <ul> <li>Click add field</li> <li> <p>In creating:</p> <ul> <li>Select type Text</li> <li>Name: Paragraph</li> <li>Long text</li> </ul> </li> <li> <p>In Appearance</p> <ul> <li>Select Multiple lines</li> </ul> </li> <li>Click create and configure</li> <li>Click Save</li> </ul> <p>Once you created both fields, you can now hit <em>save</em> on the top right corner of the content type as we finished creating our content type.</p> <h3>Contribute the first section</h3> <p>As we now have the first content type, we can contribute to creating new content. To do so, in the main contentful header, head to the tab <em>Content</em>.</p> <p>By default contentful filter with the current content type, we can create a new section hitting the button <em>Add Section</em></p> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/4516da549225bbc5a2b7e50f5a711ba6/50383/contribute_new_section.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 50.27027027027027%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAKCAIAAAA7N+mxAAAACXBIWXMAAC4jAAAuIwF4pT92AAAA40lEQVQoz2OISMkOT8wKT8yOz64IT69ILahOzK9Jyq9NK6xOzqtMLawBkvFZ5bGZZfFZQFQOR4nZZQxff/38+v37v///v/749ez9r1+/f//9B+ShACD/559/f/6BGHAEVMbwnwKAovkf2Gwg+Pz77/7bH/c8e/X81zOIzYQ1w9U9/vq7/cC7up3vdp7bf+P563//sZuAXfPPf3+nn/up1P/Detqnc48+QxxFWDNc/7ff//Y/+r3qyo+P3/+SoBlqxJ/f7959ePf+w98/v/8T6Wcky/9devTt/INvIFv/kaKZSAAAZEMonj5FkJgAAAAASUVORK5CYII='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="Contribute a new section" title="Contribute a new section" src="/static/4516da549225bbc5a2b7e50f5a711ba6/50383/contribute_new_section.png" srcset="/static/4516da549225bbc5a2b7e50f5a711ba6/1d79a/contribute_new_section.png 185w, /static/4516da549225bbc5a2b7e50f5a711ba6/1efb2/contribute_new_section.png 370w, /static/4516da549225bbc5a2b7e50f5a711ba6/50383/contribute_new_section.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <p>You end up on the form to edit the entity. You need to provide a title and you can provide some text for the paragraph. Then hit publish.</p> <p>You are done for the CMS part (keep it open for latter).</p> <img alt="Install Nuxtjs" src="/f761aed1432bcb8b3c038f88ac703dbe/built-with-nuxt.svg" style="margin-top:60px;"> <h2>Installing Nuxtjs</h2> <p>First, we are going to create a Nuxt application.</p> <p>To do so, you need to have Npm installed and type in:</p> <p><code>npx create-nuxt-app &#x3C;project-name></code>.</p> <p>Once you launch the command, you will need to answer a few questions:</p> <ul> <li>Project name: cms<em>driven</em>static</li> <li>Project description: /</li> <li>Author: /</li> <li>Package manager: / (I will use Npm)</li> <li>UI framework: you can use any of them as you like (I will not use one here)</li> <li>Custom Server framework: None</li> <li>Nuxt.js modules: Axios, Dotenv</li> <li>Linting tool: as you like, I go with and recommend all of them (ESLint, Prettier, Lint staged files, Stylelint needs some configuration hence I will not use it here)</li> <li>Test framework: I will not talk about tests here, but it’s obviously recommended to use tests (both Jest and AVA are great)</li> <li>Rendering mode: Universal (SSR)</li> <li>Dev tools: use jsconfig.json, if using vscode as your IDE</li> </ul> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/7c3741d4efe190cb8c421404d660456e/50383/install_nuxtjs.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 38.91891891891891%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAICAYAAAD5nd/tAAAACXBIWXMAAC4jAAAuIwF4pT92AAABjElEQVQoz32S607CQBBGd7ctyL2l0lgwKZRLsSBtWhExgr7/cxhfQRLifM4uGowm/phMZ5ucnm+nQjVqr7Lbheq4pFyXLN+H3W7D6nTgXFzAEQK2lJBKQVgWhG2fuz7j91z01Y+i1rKOSTpHVpSULpe4yTIE4zG80QhtLm82Q7Pfh+N5EJeXkLq7LmSvB1GvfwPPUKdpv++fd8iyjO7KkqaLBfqrFbpJAqXNw9CABFtLthbVKkSjYaCiUvltSBp4yIscw+GQZmwzYdAizzFgu4ifXQ1ptX6a/FcMrNuHoiyQJAlNp1OkaYqcDZP5HEEcI+KzgCPLq6tTVI6s71xqa237B9iw33f7J+RZTuv1mm5XKyo5+uN2S+5kQhVdvR4p7jIIiOMSg0n6PjGURLNJolY7R660neP+ZY/lYklFUYCh2Gw2eLi/R5dtBduZyBzdWOntOs7p/vgvMF3PSpHZulVVb6N4hDAMP6IoogmbxHFMPNP1YEC255FgM96qKYYTRzZ2uuvZWOsP+v7xEzA6wHHOW/JXAAAAAElFTkSuQmCC'); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="recap of the install configuration Nuxtjs" title="recap of the install configuration Nuxtjs" src="/static/7c3741d4efe190cb8c421404d660456e/50383/install_nuxtjs.png" srcset="/static/7c3741d4efe190cb8c421404d660456e/1d79a/install_nuxtjs.png 185w, /static/7c3741d4efe190cb8c421404d660456e/1efb2/install_nuxtjs.png 370w, /static/7c3741d4efe190cb8c421404d660456e/50383/install_nuxtjs.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <p>You can run the Nuxt dev server using:</p> <div class="gatsby-highlight" data-language="bash"><pre class="language-bash"><code class="language-bash"><span class="token builtin class-name">cd</span> cms_driven_static <span class="token function">npm</span> run dev</code></pre></div> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/d8d3907c7efc027102d09764338a8d11/50383/boot_nuxt.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 49.18918918918919%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAKCAIAAAA7N+mxAAAACXBIWXMAAAsSAAALEgHS3X78AAAAp0lEQVQoz6WRywrCMBBF+/+f4g+4E5dSlEKp9OGzIhY1aW2CtkkmqaEFNyEtwbvM5MwcZrzOHtVn5IPXTUW5wgPwlryWzHnyYBvSxyzwadvY5puwYowrkATY4pzOl6u4uOnXpmUAMAFzIeLdKTtekn2+jtJNmOTXIkoOfrBFZfWTsmrrMqGfO6qe+IUqgnCNS9JqHWPz3sidGHCQIAxbh1P1jf6AbfkCYmBNT4IhYwsAAAAASUVORK5CYII='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="First npm run dev Nuxtjs" title="First npm run dev Nuxtjs" src="/static/d8d3907c7efc027102d09764338a8d11/50383/boot_nuxt.png" srcset="/static/d8d3907c7efc027102d09764338a8d11/1d79a/boot_nuxt.png 185w, /static/d8d3907c7efc027102d09764338a8d11/1efb2/boot_nuxt.png 370w, /static/d8d3907c7efc027102d09764338a8d11/50383/boot_nuxt.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <p>You are done installing Nuxtjs <img alt="party popper emoji" src="/eceeb77e2efaae09977e9ca92a0464cb/party_popper_emoji.svg" style="max-height: 20px;"></p> <h2>Fetch the section content in Nuxtjs</h2> <p>We will now display the <code>Section</code> content in the Nuxt frontend. To do so, we need to install the contentful module: <code>npm install contentful --save</code>.</p> <p>Then to use the contentful package in Nuxt, we will create a plugin. Plugins are the recommended way to use javascript libraries globally.</p> <p>Finally, we will use this plugin to fetch content in the application pages.</p> <h3>Creating the Nuxt.js plugin</h3> <p>In the <em>plugins</em> directory of the project create a new file: <code>contentful.js</code></p> <p>Add the following code in the file:</p> <div class="gatsby-highlight" data-language="js"><pre class="language-js"><code class="language-js"><span class="token comment">// First we import the contentful node module</span> <span class="token keyword">const</span> contentful <span class="token operator">=</span> <span class="token function">require</span><span class="token punctuation">(</span><span class="token string">'contentful'</span><span class="token punctuation">)</span> <span class="token comment">// Those are set via `env` property in nuxt.config.js or environment variables</span> <span class="token keyword">const</span> config <span class="token operator">=</span> <span class="token punctuation">{</span> space<span class="token operator">:</span> process<span class="token punctuation">.</span>env<span class="token punctuation">.</span><span class="token constant">NUXT_ENV_CONTENTFUL_SPACE</span><span class="token punctuation">,</span> accessToken<span class="token operator">:</span> process<span class="token punctuation">.</span>env<span class="token punctuation">.</span><span class="token constant">NUXT_ENV_CONTENTFUL_ACCESS_TOKEN</span> <span class="token punctuation">}</span> <span class="token comment">// Create a client to setup fetching content</span> <span class="token keyword">const</span> client <span class="token operator">=</span> contentful<span class="token punctuation">.</span><span class="token function">createClient</span><span class="token punctuation">(</span>config<span class="token punctuation">)</span> <span class="token comment">// Our first method to fetch all section content type</span> client<span class="token punctuation">.</span><span class="token function-variable function">getSectionContent</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> client<span class="token punctuation">.</span><span class="token function">getEntries</span><span class="token punctuation">(</span><span class="token string">''</span><span class="token punctuation">,</span> <span class="token punctuation">{</span> content_type<span class="token operator">:</span> <span class="token string">'section'</span> <span class="token punctuation">}</span><span class="token punctuation">)</span> <span class="token keyword">export</span> <span class="token keyword">default</span> <span class="token punctuation">(</span><span class="token parameter"><span class="token punctuation">{</span> app <span class="token punctuation">}</span></span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token comment">// Add the function directly to the context.app object</span> app<span class="token punctuation">.</span>contentfulClient <span class="token operator">=</span> client <span class="token punctuation">}</span></code></pre></div> <p>Now we will register the plugin in our Nuxt config, head to the file <code>nuxt.config.js</code>. In this file you will find an empty plugins array. Add <code>{ src: '~/plugins/contentful' },</code> in this array:</p> <div class="gatsby-highlight" data-language="diff"><pre class="language-diff"><code class="language-diff"><span class="token deleted-sign deleted"><span class="token prefix deleted">-</span> plugins: [], </span><span class="token inserted-sign inserted"><span class="token prefix inserted">+</span> plugins: [{ src: '~/plugins/contentful' }],</span></code></pre></div> <p>That's all for the plugin configuration. <img alt="rocket emoji" src="/d418f53428b316e7d3ac125bf117dd76/rocket_emoji.svg" style="max-height: 20px;max-width: 20px;"></p> <p>Finally, we need to specify the contentful credentials, to do so, we will use the dotenv module:</p> <ul> <li>In <code>nuxt.config.js</code> add <code>'@nuxtjs/dotenv'</code> in the <code>buildModules</code> array:</li> </ul> <div class="gatsby-highlight" data-language="diff"><pre class="language-diff"><code class="language-diff"><span class="token unchanged"><span class="token prefix unchanged"> </span> buildModules: [ <span class="token prefix unchanged"> </span> '@nuxtjs/eslint-module', </span><span class="token inserted-sign inserted"><span class="token prefix inserted">+</span> '@nuxtjs/dotenv' </span><span class="token unchanged"><span class="token prefix unchanged"> </span> ],</span></code></pre></div> <ul> <li>Then in your <code>.env</code> file, you need to add two variables: <code>NUXT_ENV_CONTENTFUL_SPACE</code> and <code>NUXT_ENV_CONTENTFUL_ACCESS_TOKEN</code>:</li> </ul> <p>Using the prefix <code>NUXT_ENV_</code> enables Nuxtjs automatic injection of environment variables into <code>process.env</code>.</p> <p>To fill in those values go back to contentful. Go to the settings tab and find ‘api keys’. If one already exists, use it, else create a new one. (specify any name and description)</p> <p>Use both <code>Space ID</code> and <code>Content Delivery API - access token</code> to fill in your <code>.env</code> variables.</p> <p>Restart your server (interrupt and <code>npm run dev</code>), the webpage should load errors free.</p> <p>You should be set to fetch your content on your Nuxtjs page.</p> <h3>Display the section content</h3> <p>Let’s make this API call and display the section content.</p> <p>Head to the file <code>pages/index.vue</code>. It’s the entry-point of your nuxtjs application, namely the landing page.</p> <h4>A brief introduction to Vuejs single file component syntax</h4> <p>A single file Vue.js component is organized in 3 parts:</p> <div class="gatsby-highlight" data-language="html"><pre class="language-html"><code class="language-html"><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>template</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>p</span> <span class="token attr-name">class</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>paragraph<span class="token punctuation">"</span></span><span class="token punctuation">></span></span>{{ greetings }} World!<span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>p</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>template</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>script</span><span class="token punctuation">></span></span><span class="token script"><span class="token language-javascript"> module<span class="token punctuation">.</span>exports <span class="token operator">=</span> <span class="token punctuation">{</span> <span class="token function">data</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">return</span> <span class="token punctuation">{</span> greetings<span class="token operator">:</span> <span class="token string">'Hello'</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> </span></span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>script</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>style</span> <span class="token attr-name">scoped</span><span class="token punctuation">></span></span><span class="token style"><span class="token language-css"> <span class="token selector">.paragraph</span> <span class="token punctuation">{</span> <span class="token property">font-size</span><span class="token punctuation">:</span> 2em<span class="token punctuation">;</span> <span class="token property">text-align</span><span class="token punctuation">:</span> center<span class="token punctuation">;</span> <span class="token punctuation">}</span> </span></span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>style</span><span class="token punctuation">></span></span></code></pre></div> <ul> <li>The template part which is the rendered HTML for the component</li> <li>The script part which contains the component logic</li> <li>The style part which is, no surprise, the CSS style applied to the component.</li> </ul> <h4>Fetching the section content</h4> <p>First, we are going to fetch the section data and we want to fetch it server-side to be able to render the full HTML before sending it to the client.</p> <p>To do so add an <code>asyncData</code> method below the component in the script HTML element, ending with the export looking like this:</p> <div class="gatsby-highlight" data-language="js"><pre class="language-js"><code class="language-js"><span class="token keyword">export</span> <span class="token keyword">default</span> <span class="token punctuation">{</span> components<span class="token operator">:</span> <span class="token punctuation">{</span> Logo <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token keyword">async</span> <span class="token function">asyncData</span><span class="token punctuation">(</span><span class="token parameter">context</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">const</span> sections <span class="token operator">=</span> <span class="token keyword">await</span> context<span class="token punctuation">.</span>app<span class="token punctuation">.</span>contentfulClient<span class="token punctuation">.</span><span class="token function">getSectionContent</span><span class="token punctuation">(</span><span class="token punctuation">)</span> console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span>sections<span class="token punctuation">.</span>items<span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token punctuation">{</span><span class="token punctuation">}</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span></code></pre></div> <p>Reload your page <code>http://localhost:3000/</code>, you should see your Section item in the console.</p> <p>Then we will add some logic to the contentful plugin to map the interesting properties for the landing page Vuejs component:</p> <div class="gatsby-highlight" data-language="js"><pre class="language-js"><code class="language-js"><span class="token comment">// First we import the contentful node module</span> <span class="token keyword">const</span> contentful <span class="token operator">=</span> <span class="token function">require</span><span class="token punctuation">(</span><span class="token string">'contentful'</span><span class="token punctuation">)</span> <span class="token comment">// Those are set via `env` property in nuxt.config.js or environment variables</span> <span class="token keyword">const</span> config <span class="token operator">=</span> <span class="token punctuation">{</span> space<span class="token operator">:</span> process<span class="token punctuation">.</span>env<span class="token punctuation">.</span><span class="token constant">NUXT_ENV_CONTENTFUL_SPACE</span><span class="token punctuation">,</span> accessToken<span class="token operator">:</span> process<span class="token punctuation">.</span>env<span class="token punctuation">.</span><span class="token constant">NUXT_ENV_CONTENTFUL_ACCESS_TOKEN</span> <span class="token punctuation">}</span> <span class="token comment">// Create a client to setup fetching content</span> <span class="token keyword">const</span> client <span class="token operator">=</span> contentful<span class="token punctuation">.</span><span class="token function">createClient</span><span class="token punctuation">(</span>config<span class="token punctuation">)</span> <span class="token comment">// Our first method to fetch all section content type</span> client<span class="token punctuation">.</span><span class="token function-variable function">getSectionContent</span> <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> client <span class="token punctuation">.</span><span class="token function">getEntries</span><span class="token punctuation">(</span><span class="token string">''</span><span class="token punctuation">,</span> <span class="token punctuation">{</span> content_type<span class="token operator">:</span> <span class="token string">'section'</span> <span class="token punctuation">}</span><span class="token punctuation">)</span> <span class="token comment">// map the interesting properties from section</span> <span class="token punctuation">.</span><span class="token function">then</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token parameter"><span class="token punctuation">{</span> items <span class="token punctuation">}</span></span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">return</span> items<span class="token punctuation">.</span><span class="token function">map</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token parameter">section</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token keyword">return</span> <span class="token punctuation">{</span> title<span class="token operator">:</span> section<span class="token punctuation">.</span>fields<span class="token punctuation">.</span>title<span class="token punctuation">,</span> paragraph<span class="token operator">:</span> section<span class="token punctuation">.</span>fields<span class="token punctuation">.</span>paragraph <span class="token punctuation">}</span> <span class="token punctuation">}</span><span class="token punctuation">)</span> <span class="token punctuation">}</span><span class="token punctuation">)</span> <span class="token keyword">export</span> <span class="token keyword">default</span> <span class="token punctuation">(</span><span class="token parameter"><span class="token punctuation">{</span> app <span class="token punctuation">}</span></span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span> <span class="token comment">// Add the function directly to the context.app object</span> app<span class="token punctuation">.</span>contentfulClient <span class="token operator">=</span> client <span class="token punctuation">}</span></code></pre></div> <p>And use it in the asyncData of <code>pages/index.vue</code>:</p> <div class="gatsby-highlight" data-language="js"><pre class="language-js"><code class="language-js"> <span class="token keyword">async</span> <span class="token function">asyncData</span><span class="token punctuation">(</span><span class="token parameter">context</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">const</span> sections <span class="token operator">=</span> <span class="token keyword">await</span> context<span class="token punctuation">.</span>app<span class="token punctuation">.</span>contentfulClient<span class="token punctuation">.</span><span class="token function">getSectionContent</span><span class="token punctuation">(</span><span class="token punctuation">)</span> console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span>sections<span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token punctuation">{</span><span class="token punctuation">}</span> <span class="token punctuation">}</span></code></pre></div> <h4>Display the section</h4> <p>Finally let’s display our data, first make the sections available in the state of the component:</p> <div class="gatsby-highlight" data-language="js"><pre class="language-js"><code class="language-js"><span class="token keyword">async</span> <span class="token function">asyncData</span><span class="token punctuation">(</span><span class="token parameter">context</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">return</span> <span class="token punctuation">{</span> sections<span class="token operator">:</span> <span class="token keyword">await</span> context<span class="token punctuation">.</span>app<span class="token punctuation">.</span>contentfulClient<span class="token punctuation">.</span><span class="token function">getSectionContent</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span></code></pre></div> <p>Update the template to use the new data from the asyncData:</p> <div class="gatsby-highlight" data-language="html"><pre class="language-html"><code class="language-html"><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>template</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span> <span class="token attr-name">class</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>container<span class="token punctuation">"</span></span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span> <span class="token attr-name">class</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>logo-container<span class="token punctuation">"</span></span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>logo</span> <span class="token punctuation">/></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>div</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span> <span class="token attr-name">v-for</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>section in sections<span class="token punctuation">"</span></span> <span class="token attr-name">:key</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>section.title<span class="token punctuation">"</span></span> <span class="token attr-name">class</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>section<span class="token punctuation">"</span></span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>h1</span> <span class="token attr-name">class</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>title<span class="token punctuation">"</span></span><span class="token punctuation">></span></span> {{ section.title }} <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>h1</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>h2</span> <span class="token attr-name">class</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>subtitle<span class="token punctuation">"</span></span><span class="token punctuation">></span></span> {{ section.paragraph }} <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>h2</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>div</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>div</span><span class="token punctuation">></span></span> <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>template</span><span class="token punctuation">></span></span></code></pre></div> <p>Update the style of the component:</p> <div class="gatsby-highlight" data-language="html"><pre class="language-html"><code class="language-html"><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>style</span> <span class="token attr-name">scoped</span><span class="token punctuation">></span></span><span class="token style"><span class="token language-css"> <span class="token selector">.container</span> <span class="token punctuation">{</span> <span class="token property">margin</span><span class="token punctuation">:</span> 0 auto<span class="token punctuation">;</span> <span class="token property">min-height</span><span class="token punctuation">:</span> 100vh<span class="token punctuation">;</span> <span class="token property">display</span><span class="token punctuation">:</span> flex<span class="token punctuation">;</span> <span class="token property">flex-direction</span><span class="token punctuation">:</span> column<span class="token punctuation">;</span> <span class="token property">justify-content</span><span class="token punctuation">:</span> center<span class="token punctuation">;</span> <span class="token property">align-items</span><span class="token punctuation">:</span> center<span class="token punctuation">;</span> <span class="token property">text-align</span><span class="token punctuation">:</span> center<span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token selector">.logo-container</span> <span class="token punctuation">{</span> <span class="token property">height</span><span class="token punctuation">:</span> 50%<span class="token punctuation">;</span> <span class="token property">width</span><span class="token punctuation">:</span> auto<span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token selector">.section</span> <span class="token punctuation">{</span> <span class="token property">margin-top</span><span class="token punctuation">:</span> 20px<span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token selector">.title</span> <span class="token punctuation">{</span> <span class="token property">font-family</span><span class="token punctuation">:</span> <span class="token string">'Quicksand'</span><span class="token punctuation">,</span> <span class="token string">'Source Sans Pro'</span><span class="token punctuation">,</span> -apple-system<span class="token punctuation">,</span> BlinkMacSystemFont<span class="token punctuation">,</span> <span class="token string">'Segoe UI'</span><span class="token punctuation">,</span> Roboto<span class="token punctuation">,</span> <span class="token string">'Helvetica Neue'</span><span class="token punctuation">,</span> Arial<span class="token punctuation">,</span> sans-serif<span class="token punctuation">;</span> <span class="token property">display</span><span class="token punctuation">:</span> block<span class="token punctuation">;</span> <span class="token property">font-weight</span><span class="token punctuation">:</span> 300<span class="token punctuation">;</span> <span class="token property">font-size</span><span class="token punctuation">:</span> 60px<span class="token punctuation">;</span> <span class="token property">color</span><span class="token punctuation">:</span> #35495e<span class="token punctuation">;</span> <span class="token property">letter-spacing</span><span class="token punctuation">:</span> 1px<span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token selector">.subtitle</span> <span class="token punctuation">{</span> <span class="token property">font-weight</span><span class="token punctuation">:</span> 300<span class="token punctuation">;</span> <span class="token property">font-size</span><span class="token punctuation">:</span> 21px<span class="token punctuation">;</span> <span class="token property">color</span><span class="token punctuation">:</span> #526488<span class="token punctuation">;</span> <span class="token property">word-spacing</span><span class="token punctuation">:</span> 5px<span class="token punctuation">;</span> <span class="token property">padding-bottom</span><span class="token punctuation">:</span> 15px<span class="token punctuation">;</span> <span class="token punctuation">}</span> </span></span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>style</span><span class="token punctuation">></span></span></code></pre></div> <p>Then reload your homepage, you will be able to see your new section content: <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/c211aa89a8f04a7253a6102790164fae/50383/section_display_1.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 49.18918918918919%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAKCAIAAAA7N+mxAAAACXBIWXMAAC4jAAAuIwF4pT92AAAAmklEQVQoz82RywrCMBBF8/+f4taN7vwGKVR0UdSWJiaTNrEK5jF1Om4FKSI4XEJyOWc2EeMXI34lJ8Q04mwZcXKKq17utjnjDPnFOgyrqlysN6Vq6ZnxzQoRYwohUh58UmLKw3BvlD6c6mMt99XZ9p5KZtKEkcIRvbtpcMY66LyxHPB0p1Iq20iQl44Bb7g04MB6YFL85Vd9nCd6MkymzzFS1QAAAABJRU5ErkJggg=='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="Display our first section from cms" title="Display our first section from cms" src="/static/c211aa89a8f04a7253a6102790164fae/50383/section_display_1.png" srcset="/static/c211aa89a8f04a7253a6102790164fae/1d79a/section_display_1.png 185w, /static/c211aa89a8f04a7253a6102790164fae/1efb2/section_display_1.png 370w, /static/c211aa89a8f04a7253a6102790164fae/50383/section_display_1.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <p>Congratulations <img alt="party popper emoji" src="/eceeb77e2efaae09977e9ca92a0464cb/party_popper_emoji.svg" style="max-height: 20px;"></p> <p>Let’s add one more section content in Contentful to check that our page works correctly.</p> <p>Once you add the new section content and publish it you should see both sections on your homepage: <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/c683308a50a6d960e2c7431f294720ac/50383/section_display_2.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 49.18918918918919%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAKCAIAAAA7N+mxAAAACXBIWXMAAC4jAAAuIwF4pT92AAAA/UlEQVQoz6VSy2rDMBDU//9H/6M9lBYCvZWG0JDED8l6eiXLTms93LWNL6WFlizDsBpm2DmITDcM+VHNC7fxQ47Dv8Nrfmfo3dMjXIflmf8czpP5HB5Oh/vnlz2jv4adH4z14HpAtn1rPSoauv178Xo4Hc/07XBhXHd+AOdXz2x2vXU9EdrWjWbccNlWTBe1vFSipGpRLBPQCEAD6ohZF4byGY1s59oZZ6mRUh7HGGLy/ZVLUzHBpS4pb8HFmMYxIOetPoaIgU4oQEhthbLI0jiuoGKqqMW55MeC4TVlnNIODatzBcFr3xBzmqvkKYQYQliPJJy8ebaF3PJJvgDcY0LhYOQR8wAAAABJRU5ErkJggg=='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="Display both sections from cms" title="Display both sections from cms" src="/static/c683308a50a6d960e2c7431f294720ac/50383/section_display_2.png" srcset="/static/c683308a50a6d960e2c7431f294720ac/1d79a/section_display_2.png 185w, /static/c683308a50a6d960e2c7431f294720ac/1efb2/section_display_2.png 370w, /static/c683308a50a6d960e2c7431f294720ac/50383/section_display_2.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <img alt="Netlify logo" src="/50b3f93d034be3b993615e298e9633dc/netlify-logo.svg" style="margin-top: 60px;width:90%;"> <h2>Deploy the website using Netlify</h2> <h3>Setting-up automatic deployment</h3> <p>Finally, we will deploy the website, making it available on the Internet.</p> <p>First of all, create a repository for the project on Github or Gitlab and push your project.</p> <p>Once you are there, you can head to Netlify website: <a href="https://www.netlify.com/">https://www.netlify.com/</a>, create an account using your favorite authentication: <a href="https://app.netlify.com/signup">https://app.netlify.com/signup</a></p> <p>Once on the project page, I recommend using ‘New site from Git’ to setup up a static website deployment with the added benefit of an out of the box working CI, building your site with every new commit on your production branch. <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/d75ec35f2682eb76748ba47dda5e719c/50383/netlify_home.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 51.891891891891895%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAKCAIAAAA7N+mxAAAACXBIWXMAAC4jAAAuIwF4pT92AAABbklEQVQoz2OQMLVTMbFUM7VUM7FSM7YUU9Xml1cVUFADkhAGGgIK8sqqKBiYuoRGMPDKKLOKyUAQi5g0i6g0iA0hxWQY+EUZBMSQEaeYrICYLK+INKeEHEP35GnL121YtGL1klVrlqxaC0SLwYylq0EImbF49doVK1YVt7d7d7QlTZ/qFhnLcPfBg//////9+/c/IfAPrO7ykYPL9mw/evlcW28/w7UbN86eOXP/4cMPnz69//Dxy9evcPTp8+enL148e/Hy6bPnL16++gIEX7/++fHz//cf/7/9eP/hA8PFK1dWr1v34OHD799/fP367ScS+P79++u3b1+/efvqzZt379//gAj+/AlU9PXHj5+/fzM8fvL03IULb96+BXJ//PjxBwn8/v376zcQAJPf/6CCH79+Mfz79w/db//+/YUBsFeB8B9mEPz684cBqOIfmgH//gHtBLoC6E6Qa37+/PvnL17NqABLUGMoAGoGAFzsxVDz+9JwAAAAAElFTkSuQmCC'); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="Netlify project selection page" title="Netlify project selection page" src="/static/d75ec35f2682eb76748ba47dda5e719c/50383/netlify_home.png" srcset="/static/d75ec35f2682eb76748ba47dda5e719c/1d79a/netlify_home.png 185w, /static/d75ec35f2682eb76748ba47dda5e719c/1efb2/netlify_home.png 370w, /static/d75ec35f2682eb76748ba47dda5e719c/50383/netlify_home.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <p>Select your repository: <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/6f68e9f2fec4bca1cece357231be6862/50383/netlify_add_new_step2.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 63.24324324324324%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAANCAIAAAAmMtkJAAAACXBIWXMAAC4jAAAuIwF4pT92AAABkUlEQVQoz5VR20oCURSdfsAgysceeuon+oOI6Dt6KvOhi5kQUb8meCExGiNwwjQr56Iz+9ynNTNmGhi0WWdzOGetvfc6x1rJb1q5dWs1/w/kNta2tnf2Dqzz69ujs1LhogwULysnpQpy4eLq+LycIbuaB/ilm7ud3X0rjmMpJXJsjFbKpKGUipeHSPmom4iJcW1MSGR3u8+93pPjdAeDD99/97w3z2NCaIWyaWhsNXEO1WHxNBFDiayMDjjzGZHWY876o5EbhRMpIp6EEGLW2aT8h07Hmh1pbYiIM6iJTcFw+Mf81qwS49zpvfb6g5ck9wFsXN9HU1xlyJhZwM5C54hYREkwzqfDSomFmTPL82Ic/Yjh2Q8Cd+QiZg7BxvCwkjzW4hdMxVm9secNXc8nCtBZKZKSa61xm1Kxj9B5mZjaba9ajVotv9EY1RuftXpw31Kdjnp8BLRtK9vW6SfFKX9hbJgWQSAmIXOcsFYLm01kORzGsEAUM5bgl+fs3wEJGAMkj4S3SoFDGP3BNxlgUn4Bk06dkamLwYYAAAAASUVORK5CYII='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="Select your repository to publish with Netlify" title="Select your repository to publish with Netlify" src="/static/6f68e9f2fec4bca1cece357231be6862/50383/netlify_add_new_step2.png" srcset="/static/6f68e9f2fec4bca1cece357231be6862/1d79a/netlify_add_new_step2.png 185w, /static/6f68e9f2fec4bca1cece357231be6862/1efb2/netlify_add_new_step2.png 370w, /static/6f68e9f2fec4bca1cece357231be6862/50383/netlify_add_new_step2.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span> If it doesn't appear in the list click on <em>Configure the Netlify app on Github</em></p> <p>For your site settings, go for:</p> <ul> <li>Base directory: not set -> useful if you use a mono repository to set the directory where you can build the front</li> <li>Build command: <code>npm run generate</code></li> <li>Public directory: <code>dist</code></li> <li> <p>Click <em>show advanced</em> to add environment variables matching the ones defined in .env:</p> <ul> <li><code>NUXT_ENV_CONTENTFUL_SPACE</code></li> <li><code>NUXT_ENV_CONTENTFUL_ACCESS_TOKEN</code></li> </ul> </li> </ul> <p><span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/cd2b6d86079bc7ee1e3759cfddc9f82f/50383/netlify_add_new_step_3.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 85.94594594594595%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAARCAYAAADdRIy+AAAACXBIWXMAAC4jAAAuIwF4pT92AAABt0lEQVQ4y51Uy07DMBDsF3Pik/gKxAVx5gRCHItEXk0b23k4T3vZcR5N3VZVibRd243HM7sTb4wx78SPtdZwWLrjwet2fAzmjPWxOf43YmVCUBDFFMYJ7dI9RUnCseO1iIIwoojXf8OQ4l1KgzEn+PjZMKpZIdpKa1J5TnlRUsExjoslj1FS23Y0kbD2ODAAtGuKWtcklXIAbdedyLsme2YHjM1EcNlQVhWDFW4+DAPq4qLvezef6rbEei/yGWDbtlQzy6auHZA9Fv8iSx/4DDDlpvxwA7YcgmU3TePKoPmAmgM11hzI/t6LgJIlRwyace7MKHGR/R/JkKqkWiSjbsOUUce7JUPa3F2fDUD9rt8EFFI684ZxTEmaOlY4AM3qOCMuSb0KWFXa+XBflpRxU5TKqShLZ6XZ2LCWv+8qYAdGzKaF72AbXkczwNJM87lJcwl8hidfCuwAqfvDgZnmo1QGx3cthKQGh3HATmDtAdozQHRXZoK6unHyZ78hQ7qefAhAgK/3OkD/tgmloK8woKDIqZ9PvnGF+bdNhMEwMrVP35/08PpMj28vtFXSvWk8+3jGXt82hz+eyDZnsL+SvQAAAABJRU5ErkJggg=='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="Nuxt settings with Netlify" title="Nuxt settings with Netlify" src="/static/cd2b6d86079bc7ee1e3759cfddc9f82f/50383/netlify_add_new_step_3.png" srcset="/static/cd2b6d86079bc7ee1e3759cfddc9f82f/1d79a/netlify_add_new_step_3.png 185w, /static/cd2b6d86079bc7ee1e3759cfddc9f82f/1efb2/netlify_add_new_step_3.png 370w, /static/cd2b6d86079bc7ee1e3759cfddc9f82f/50383/netlify_add_new_step_3.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <p>Your site is now online! <img alt="party popper emoji" src="/eceeb77e2efaae09977e9ca92a0464cb/party_popper_emoji.svg" style="max-height: 20px;"></p> <p>You can find the Netlify generated URL (ending in <em>.netlify.com</em>) on the Netlify project overview. <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/0bcf2914f645610097dad5e35a216997/50383/netlify_site_live.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 48.64864864864865%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAKCAIAAAA7N+mxAAAACXBIWXMAAC4jAAAuIwF4pT92AAABxElEQVQoz22PW08TQRTH10SrJEZphS1qqy2FtmB98MEHE+sl8UGEcokphBgDQYFXXnjwUxlJ9NsYNSSmdrud7u7sXM5c6tldUB+c/DJz5sz/f84ZpzC7UJhtFqqNfL11vdq4Vq5dLdXylXqxfv9GrXnl5l0UTM0lmny14c7faz5qT80tTlYabr3l5Nxyrli+UJgptR5svT/c2HnX3TvY3Dvo7u53tt8udbcnK/OXZ+7k3NIZxfIlt3Rx+vbErYrTXl5/1tl4+GLp6PiDd3r68/s30uuNraWUfvx0cvL5y8rWm8cvO6h5urKOPFlOaL9ae7762gkZj6XMCKIootQfkSCOmZQ9f9AnQ8IY4ZxwlhEKkYmpEE4sQWgttaYAX/v9H0PfE7zPYsSXwhfc4wwZCD5I8z5j0hi0cKUcKiQeTCmptDVGSWmNHadLGYNFs9haq5XKkihOLAD/mLUGAH9IfEK00qDUMKZeFHEAY0wQhr88D3cwOjP/7YwI9AMwIXDXxmidoNIkaKympVKYx/j/ZiyMbSnn2ajGWpVUQYsZn68/YyfmiAucPgbA8RjnMef4hlqbgtdREARhNApCSP+Pf0AxS/kNL5DG0kvgfxoAAAAASUVORK5CYII='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="Access your netlify website" title="Access your netlify website" src="/static/0bcf2914f645610097dad5e35a216997/50383/netlify_site_live.png" srcset="/static/0bcf2914f645610097dad5e35a216997/1d79a/netlify_site_live.png 185w, /static/0bcf2914f645610097dad5e35a216997/1efb2/netlify_site_live.png 370w, /static/0bcf2914f645610097dad5e35a216997/50383/netlify_site_live.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></p> <h3>Using your own domain name</h3> <p>Here again, Netlify eases the process, you simply need to <a href="http://letmegooglethat.com/?q=how+to+register+a+domain+name">have a domain name registered</a>.</p> <p>Once you have a domain name, you can carry on the settings:</p> <ul> <li>Click on <em>Set up a custom domain</em> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/99f857bc44153ee6fe20000e6ad09947/50383/netlify_add_domain_step_1.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 48.64864864864865%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAKCAIAAAA7N+mxAAAACXBIWXMAAC4jAAAuIwF4pT92AAABwElEQVQoz22PTW/TQBCGjQQBJARNKE6BGJKmTQKEAwcOSASKuBTaQAtqqwpVgkr0zJUDvwlRCf4NBy5RGsfZ2F7v7NdsGMcgemD1aLwzft+ZWa+yfKey3Kk02uVW90qjczloXgqa5Xqr2rp/tdm5cP02CRZXck250fZX73Ue9RZX7i7U236r65X8oFQNzlSWat0He4dH2wcfdt5/3D08otjfP3ixs79QXz2/dKvk1/5QDc75tbPXbl68Ufd6G1tr/TcPn69/+vwlnLLBeMw4n81mnPOv346Pv//Y3Hv3eL2/1t9+urlFPNnI6b18/ezVWy8RkClVEAuRShmlaZwJodQwGo/YhAnBABiIgkTKQsyl9DKlpbXKWq7Vz9HJrygKQYxERkQSIgBKibEEIi9SW0SygDEel4o+whhlrEM0Sjl0s/kxzpGuuDvnrDF5EZHEuUXrU2ZrtdbRhEWMWYtaiMlwGA4GkHF0Lk6SkzCkqNEW5n+TCUl+rYWUFC2itXkzQ6lS2lpDTzOG6nT/v5ka01gOUKyK+eZIBvy7/Om1c3MKkrbPtAYaC5AB0D/SujmUTuM4TtJpnNBMMmtEEos5vwGVscbinzAkjgAAAABJRU5ErkJggg=='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="Head to set up a custom domain" title="Head to set up a custom domain" src="/static/99f857bc44153ee6fe20000e6ad09947/50383/netlify_add_domain_step_1.png" srcset="/static/99f857bc44153ee6fe20000e6ad09947/1d79a/netlify_add_domain_step_1.png 185w, /static/99f857bc44153ee6fe20000e6ad09947/1efb2/netlify_add_domain_step_1.png 370w, /static/99f857bc44153ee6fe20000e6ad09947/50383/netlify_add_domain_step_1.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></li> <li>In custom domain, click <em>Add a domain alias</em> and fill in the input with your domain name <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/abf189f4bad50beac94fedeb91507599/50383/netlify_add_domain_step_2.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 56.21621621621622%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAALCAIAAADwazoUAAAACXBIWXMAAAsSAAALEgHS3X78AAABhElEQVQoz31QTUvDQBTMH7fipRVFWxut2Kb0S6rgRTwKeheVoq0i2CJeRfKx3/t2k/i6qaWoOExesnk7O2/H8xvt6kFQPWyXq4318k5ps7ZWqZUquxvbdfyz5Tf3jroF/VZ/v9n124P9oHsQDOqtgdcIOkednt8MLq+uDWjBuRQyz/M4jqez2eN43B2e9hzxo3Pi6vGw1R+cnV94Nw+T0dPz7ePk9e2dMJ4QGsaxkArry+sUeTee/OD9+AnraPLsQZrlDmDThFKbpkzKjzAUAMpamy26v6EAvERICSCMoUJ+foa40gBxklDGwFgwpiCek2YZUhnDAaQxMWMekUpZgwuhtVJqPoIxESERJUwgMAHpzofCUFuLTjhUwrkTm7mYKy1cVOgSURpxJrVGM2sNKvGFMChzm1fEzhkP50LgJcFaQgillDsQkoQRRihXxXMzrVfE38Q2Y5xx7qaWSmvcguaZA45dbBN/irGtlF5ecolFYP+LcX6MmuLDMAddTIt1GdhS/AUcSFX88zeBxgAAAABJRU5ErkJggg=='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="Add a domain alias" title="Add a domain alias" src="/static/abf189f4bad50beac94fedeb91507599/50383/netlify_add_domain_step_2.png" srcset="/static/abf189f4bad50beac94fedeb91507599/1d79a/netlify_add_domain_step_2.png 185w, /static/abf189f4bad50beac94fedeb91507599/1efb2/netlify_add_domain_step_2.png 370w, /static/abf189f4bad50beac94fedeb91507599/50383/netlify_add_domain_step_2.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></li> <li>Then you will need to point your domain to netlify servers, to do so, use the provided CNAME record in your DNS provider interface <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/4c7c7b59ad1a8175f086cd8d151874ab/50383/netlify_add_domain_step_3.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 56.21621621621622%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAALCAIAAADwazoUAAAACXBIWXMAAAsSAAALEgHS3X78AAAB3ElEQVQoz32RW2/SYBiA++O8Z/PCXWxggOzQFloYtIWWAXOZoy0iMFZaKKWAzMOFLjFGF6L+A+NkG1HRTafRRKeRg45A2V62BHVBvzwX75e8z3v4PkRS9KScTyh6dE0VEzKfzESSGWFViaayMUlLKIWUWkypJSCtleVcSdLX0/kbsrYuF24hcUlJypl4WsmXb1aePH1YefxgswLB/Uebd+5t3L67IeW0NXWAohcUvZjRi7KmS7l8WtMRk9k+ZrGZzNZL9tnLGGnFXXMeBqP8OOW3E26b023GCDNKTKHOcUibtJqmBoxb7BcuTiAEGyQ5IESywRmXd1GINVvtRrPV7fWO/z5XonGM5lyBMCS7FxZtpAdBKRajT6H80GfhqvCy/mZre+d5dfvZi+rWzm51t1Z79frjp8+hFRGGwhkOpVmHL2DByN8yTrNz8zRkZAulJTEWia/yQCIFQSgiqqVycEXAvL5/ytMu75J4vXPUgdPv98+NHeajUN0xWma4M/nw2/cvXw8HmzeasPyPRqPVakM56Dz7P9lNhYVrv7rdttH7aRgdwzjf2cMMZGqUPDNPs8t8/e1efW9///3Bu4MPQ+DKLfOo1z+68xCogp/+s4NmnQxH+AJ/Ak8zlE8ALKGFMDMq4sIAAAAASUVORK5CYII='); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="Use the provided CNAME record" title="Use the provided CNAME record" src="/static/4c7c7b59ad1a8175f086cd8d151874ab/50383/netlify_add_domain_step_3.png" srcset="/static/4c7c7b59ad1a8175f086cd8d151874ab/1d79a/netlify_add_domain_step_3.png 185w, /static/4c7c7b59ad1a8175f086cd8d151874ab/1efb2/netlify_add_domain_step_3.png 370w, /static/4c7c7b59ad1a8175f086cd8d151874ab/50383/netlify_add_domain_step_3.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></li> <li>After a few minutes to hours depending on your DNS provider, you should be able to access the website using your domain name.<img alt="party popper emoji" src="/eceeb77e2efaae09977e9ca92a0464cb/party_popper_emoji.svg" style="max-height: 20px;"> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/c3527c786b25f2714349a246d443c63d/50383/netlify_add_domain_step_4.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 56.21621621621622%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAALCAIAAADwazoUAAAACXBIWXMAAAsSAAALEgHS3X78AAABX0lEQVQoz5VQ207CQBDtf2saEUKRBguI0a/wE4wPJj7oow/CQ4XtBaEXaOlud7fdrbMtJF540MnJ7M7OOWcmq7UNSzcuT7uDk46pdwd6b6z3hnrPApz1Ry1z3ILcH52bV+eDiWGO29a1MZxcDG871o3GOa+kBEgpqkpW/wmNFSUc8nuIQxaVFAJwNKTGlfirUO0BMnB8if0nZDfWP2Y2tVaU4ldDtSJB7+3p3cNjSLKjeiXOKS9LUexRFkUJvJyyKXKf32avM3s6d/KcwZJ7wgGg0hw/XHiB4wXuMkTeeu6uFzXmaOX5ketHCwfKAHmBagHTB4SOF/qrGCYzNVlZCcp4RmiG8yTFYZx8BHEQJetwAyXJGbwzxpvtFGDyJsHQ2+5wuiMJIMVplgfRFrkrG/nvaDmzHTCqu3ibZIqww8CEFy3NCLhiQgmh+AD4CNiosavvXBEaWp2by5Hf/nt8AoBYYu701X8yAAAAAElFTkSuQmCC'); background-size: cover; display: block;" ></span> <img class="gatsby-resp-image-image" alt="Access your website using your domain" title="Access your website using your domain" src="/static/c3527c786b25f2714349a246d443c63d/50383/netlify_add_domain_step_4.png" srcset="/static/c3527c786b25f2714349a246d443c63d/1d79a/netlify_add_domain_step_4.png 185w, /static/c3527c786b25f2714349a246d443c63d/1efb2/netlify_add_domain_step_4.png 370w, /static/c3527c786b25f2714349a246d443c63d/50383/netlify_add_domain_step_4.png 740w" sizes="(max-width: 740px) 100vw, 740px" style="width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;" loading="lazy" /> </a> </span></li> </ul> <h3>Using TLS (HTTPS)</h3> <p>Once you can access the website using your domain, you will want to use https for security, SEO and performance purposes, you can and should use netlify https functionality:</p> <p>Head to <code>https</code> in domain settings of your website:</p> <ul> <li>Then click on <em>Provision certificate</em> <span class="gatsby-resp-image-wrapper" style="position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 740px; " > <a class="gatsby-resp-image-link" href="/static/bfdb1c40d49c723dcb728c16d020d96d/50383/netlify_add_https_step1.png" style="display: block" target="_blank" rel="noopener" > <span class="gatsby-resp-image-background-image" style="padding-bottom: 56.21621621621622%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAALCAIAAADwazoUAAAACXBIWXMAAAsSAAALEgHS3X78AAABmklEQVQoz3VRy07CQBTtt5igMa6MoiLBtQs/CUE0AgJiRWJc+hn+hhsTjU/YKMVQaOd1Z9p6ptVEo96cTu/cuec+ncMjd6/ZrTZPdg87lfpxueGW691K46TadPdabq3dq7VOa63efudsv9u