How we saved time (and money!) on continuous integration

Hi, I’m Jack from the Datawrapper app team, and today I’ll be showing the steps we took to make our continuous integration workflows 80% faster.This post dives into DevOps and CI, so if you're familiar with those concepts you'll feel right at home!

Towards the end of last year we started to get a feeling you might recognize. We were spending too much time waiting forcontinuous integration (CI) tasks to complete, often even longer than it had taken to make the change in the first place.

There are 56 CI workflows in our monorepo, but one runs particular often: the CI for our main SvelteKit web app. This workflow is triggered between 50 and 100 times every workday, and each run was taking approximately 17(!) minutes.

The workflow is made up of 7 steps:

As you can see, we were spending almost 5 minutes just installing and building dependencies. But with a little effort, we were able to reduce the time of our CI workflows by 80% and the cost by 45%. These are the four steps that got us there:

Step 1: Switch from NPM to PNPM
Step 2: Cache dependencies and builds
Step 3: Split workflows
Step 4: Switch from Github runners to Blacksmith runners

Step 1: Switch from NPM to PNPM

Our code is organized in a monorepo and we’d been using NPM as our package manager. Unfortunately, NPM was leading us to install the same dependencies over and over across different packages of the monorepo.

Take the following simplified example: We have a big App package containing several visualization packages, each with the code for a different visualization type or family (line charts, tables, maps, etc…). Each of those visualization types contains a color legend package as well.

With NPM, each of these packages was independently installing its own version of Svelte. We could save ourselves a lot of work and disk space by installing each dependency only once:

Conveniently, this is exactly how PNPM works. It also speeds up installs by simultaneously resolving, fetching, and linking dependencies. After migrating our monorepo to PNPM, setting up the environment for CI became 74% faster:

Step 2: Cache dependencies and builds

Our raw installation speed was now much faster! But we could save even more time.

In general, dependencies only rarely change between PRs. If we change one line of a component between CI runs, ideally we’d just reuse its dependencies instead of reinstalling and rebuilding them every time.

Thankfully, PNPM downloads all dependencies to a global store and symlinks them into the different packages of our repo instead of keeping them in a bunch of spread out node_modules as NPM does. This makes caching them much easier:

name: 'Setup node environment'
description: 'Setup node and pnpm'
runs:
    using: 'composite'
    steps:
        - name: Install pnpm
          uses: pnpm/action-setup@v4

        - uses: actions/setup-node@v4
          with:
              node-version-file: '.nvmrc'

+       - name: Get pnpm store directory
+         shell: bash
+         id: pnpm-store
+         run: echo "path=$(pnpm store path)" >> "$GITHUB_OUTPUT"

+       - name: Cache pnpm store
+         uses: actions/cache@v4
+         with:
+             path: ${{ steps.pnpm-store.outputs.path }}
+             key: >-
+                 pnpm-${{ runner.os }}-${{ github.workflow }}-${{ github.job }}-${{
+                     hashFiles('pnpm-lock.yaml')
+                 }}
+             restore-keys: |
+                 pnpm-${{ runner.os }}-${{ github.workflow }}-${{ github.job }}-
+                 pnpm-${{ runner.os }}-

To cache dependencies, we run pnpm store path to get the location of the PNPM store, then pass it as the path argument to the actions/cache action and use a hash of the lockfile as the cache key. This tells the action to save the contents of the PNPM store at the end of the CI run and to reuse it for as long as the pnpm-lock.yaml file hasn't changed.

We can do something similar for our shared libraries. After extracting the build steps to a reusable composite action, we cache the result of the build based on both the contents of pnpm-lock.yaml and the source code of the library.

name: 'Build and cache libs'
description: 'Builds `shared` libraries and caches the results.'

runs:
    using: 'composite'
    steps:
        - name: Cache build artifacts for shared
          id: cache-shared
          uses: actions/cache@v4
          with:
              path: libs/shared/dist
              key: >-
                  ${{ runner.os }}-build-shared-${{hashFiles(
                    'pnpm-lock.yaml',
                    'libs/shared/**',
                    '!libs/shared/dist/**'
                  )}}

        - name: Install workspace root dependencies
          shell: bash
          run: pnpm -w install

        - name: Install shared dependencies
          shell: bash
          run: pnpm --filter @datawrapper/shared install

        - name: Build shared
          if: steps.cache-shared.outputs.cache-hit != 'true'
          shell: bash
          run: pnpm --filter @datawrapper/shared run build

So how are CI times looking now?

By switching to pnpm and caching installs and builds, we’ve cut our time to set up the environment down to just 35 seconds — an 87.5% improvement!

Step 3: Split workflows

A big reason for the remaining run time was that our workflow still ran each step in succession. Originally, the setup already took so long that parallelizing things wouldn't have made a difference.

But with the setup now being nice and speedy, we can afford to run it multiple times, which lets us run each of these jobs in parallel.

That gets us a 67% improvement in real time, or how long it takes for the longest job to finish. But in billable time it still comes to around 18 minutes, back up to about the same as our original setup without caching.

So we’ve traded 67% time savings for 39% more cost.

Is there any way to get that money back?

Step 4: Switch from GitHub runners to Blacksmith runners

Enter Blacksmith! A service that claims to be a drop-in replacement for GitHub Actions, but twice as fast and twice as cheap.

Switching to Blacksmith was as easy as promised — we logged in with GitHub, added our repository, and replaced all our runners and caches with Blacksmith’s versions.

- runs-on: ubuntu-latest
+ runs-on: blacksmith-4vcpu-ubuntu-2204

- uses: actions/cache@v4
+ uses: useblacksmith/cache@v5

Jobs run 22.14% faster on Blacksmith compared to Github Actions on average. Not quite the promised 2x speed up, but definitely a big improvement. The cost difference is even more significant — over the course of a month we spend 45% less on CI after switching to Blacksmith.

Less waiting, more building

While our fencing skills have taken a serious hit, these changes have made a huge difference to how fast we can go from creating a PR to merging it. With an 80% reduction in CI times and a significant cut in costs, we're now spending less time waiting and more time building. If your CI runs feel like an endless waiting game, I hope these changes help you shave off some precious minutes.

Now that we've sped up everything else, linting is the slowest part of our workflow. If you’ve got any tips to speed up linting in large Typescript monorepos — please let us know at hello@datawrapper.de! I want to thank my coworkers Pascal and Toni for their collaboration on the migration to PNPM as well as our team lead Marten for reviewing drafts of this blog post.