Does stacking make us more productive?

Every now and then, we may find ourselves a little pressed for time when it comes to writing the next Weekly Chart. On those occasions, it can be helpful to dig out an old Weekly and see if there’s a fresh angle to the story or data. That’s exactly what I did this time.

A while back, I wrote a Weekly Chart about collaboration on the app team, where I looked at the number of pull requests created, as the team was growing. We haven’t added any new members to the team since then, but a few things have changed regardless — most notably, we’ve adopted the stacking workflow.

So what is a stacking workflow? In a traditional development workflow, you create a pull request, wait for a teammate to review the changes, and then merge it into the main codebase (known simply as main) before moving on to the next task.

The stacking workflow changes that rhythm. Instead of waiting for changes to land on main, you stack new changes on top of previous ones. This means you can submit a change for review and immediately keep working on the next. Once the reviews come in, you can merge the whole stack of changes at once. Tools like Graphite make this a breeze, even if the reviewer requests changes to a pull request lower in the stack.

To explore how stacking has affected our contributions at Datawrapper, I looked at the number of created, merged, and closed pull requests per week. While you can see the numbers had already been growing before we adopted the new workflow (for example, because new team members joined), the number of pull requests saw a steep increase once we adopted stacking, culminating at a record high in late August of this year, as we geared up ahead of the Workspaces launch.

So, what else has changed?

Taking a look at the average number of changes for each pull request, we can see that after adopting the stacking workflow, new pull requests add and remove fewer lines of code.

That's not so surprising. Since we can stack changes on top of each other, we're encouraged to keep changes to a minimum and as self-contained as possible. You don’t have to finish the whole feature in one go, but can incrementally build up to it (for example, by first writing tests, then implementing the API endpoints, and finally the user interface).

With pull requests being smaller on average, does that mean they get reviewed and merged into the codebase more quickly?

In fact, the turnaround times of pull requests have increased, from a median of 17 hours before stacking to about 24 hours after stacking, which is the time it takes for a pull request to get merged after it was first created.

This may seem surprising at first. But thinking about it some more, it actually makes sense.

Our pull requests are still mostly reviewed by humans — we have tried out a few AI review tools, but none of them has convinced us so far — so a higher number of created pull requests naturally leads to a higher review load. A single pull request might be easier to review, but having multiple reviews lined up means that a pull request sits in the “waiting for review” stage longer. (Unfortunately, GitHub doesn't track the start time of a review, which makes the actual review time difficult to calculate.)

Also, while a single pull request in a stack contains fewer changes and should therefore be easier to review in theory, in practice, the reviewer often has to look at the whole stack of changes to get the bigger picture.

So, we’re creating more pull requests, each pull request is much smaller, and takes longer to get merged. But what does all of this say about our general productivity?

The average number of total changes (that is, additions and deletions of code) per week may give us the answer.

We went from around 40,000 added and 36,000 deleted lines of code per week before stacking to about 49,000 additions and 47,000 deletions after. That looks like a win to me — and it matches the feedback I’ve been hearing from my teammates too.

But, as usual, it’s important to take these conclusions with a grain of salt. After all, correlation doesn’t equal causation. Maybe our refactoring efforts are finally paying off, allowing us to move faster through our codebase? Or maybe — just maybe — the integration of AI into our workflows is starting to make a real impact?

All I know for sure is that we have come to love the stacking workflow here at Datawrapper and have no intention of going back.

That's it for my Weekly Chart — I hope you enjoyed this peek behind the scenes of development at Datawrapper. Until next time!