Paula Guisado & Jaime Gutiérrez, DatosRTVE: "The overall quality of our team's work has skyrocketed"

Hello everyone! So we're going to talk about our workflow that we use to deliver quick data vis to the newsroom.

This is Jaime and myself.  We work at RTVE, which is the Spanish public broadcaster. And most specifically, we work for DatosRTVE which is the data journalism unit but it's also a visualization desk. We produce our own pieces with a focus on visual information, for the digital newsroom. In this sense, Datawrapper has become a core part of our job and has allowed our graphic production to multiply. So we've come to have a really wide range of graphic production, many of them that we create regularly. But even though Datawrapper is super straightforward and allow us to create graphics quite quick, we still need time to produce them. And sometimes we need to do that even more quickly.

Steps to a faster data vis workflow

There are many times when the newsroom needs graphics almost instantly, and we acknowledge that uncertainty is inherent to journalism and especially to breaking news. We can't know beforehand exactly which charts we're going to need the following day. But we do know that there is a new cycle and some issues are more or less predictable. That's the starting point of our workflow. So the first step is identifying that there is an opportunity. Which is to be faster.

Faster with the news that we somehow can be ready for. Think about periodic statistics like unemployment or recurring issues like earthquakes or weather alerts. That's the case that you're seeing now on screen. So we usually get the alert from the news desk. Like, "hey, there's some new information about global warming."

Then the breaking news team will start working on our first draft piece. In this case, they probably even have some embargoed information. And in parallel, DatosRTVE will be working on the new data once it's available, and create the graphic. That's a graphic that we've probably done before. The news piece with the text and the photo will come out within a few minutes, but in many cases it will be without the graphic that will be included afterwards. We have identified the chance to be faster. The second step is, how usually do we have this problem? Is this happening too many times? It's also something that is very important for RTVE? Are we going to consider investing the time and effort that we need to make this thing faster?

So that's the second step, to decide if we are going to do that. Then once we know that we have room for improvement and we have decided that it makes sense for us to invest the effort, it's time to prepare. And we do that by creating a folder. We have an organized system that we constantly check, when new issues come.

And we make sure that it's easy to find for every member on the team, because the person who makes the graphics for the first time might not be the person who updates it a month later. So we pay much attention, a lot of attention to that. Then we select the graphics. Because sometimes we will need maybe two or three graphics for a topic. But other times, it will be maybe nine or even ten graphics that we want to have ready for use.

So once we decide on that we make our analysis. We understand that the coverage that we want to do. Then we create the adult templates And that's a very important part because that what's allowed us to be fast. Because once we're done, we only have to duplicate the template when needed.

After that, of course, we have to make sure that we have the communication channels ready. It's a very important part. Because if not, we would be sometimes doing graphics and nobody would know that we have them.

And we have to make sure that we are also doing the graphics that they want. We need to make sure we are following the interest of the newsroom. And it will depend on the case. In some cases, we are closely following the event and we will know that the news is ready, the data is ready before the news list. Other times, we will get the notice from them and we will start working. It will depend on the case, but we have to make sure that we are connected and we know each part of the, newsroom, the team know what we are doing.

And then, in some cases, we go one step further, and we do automation with code.  This is Jaime, and we're going to continue from here.

Automating the workflow in R

Yeah, because using templates is an effective solution,  but we detected it could be better if we worked on automation, as Paula said. It makes sense for the chart in our examples, since it's something we do every month. Basically, our Excel process could be adapted to be recurring. And all we usually did within a spreadsheet, now needs to be done with code in R thanks to the Tidyverse package and Datawrapper API.

First of all, we download data. To do that, we just need the dataset CSV link. And once we have data, we need to clean and transfer them to fit in Datawrapper's line chart data structure, that, as you know, is very specific. As you can see in the code, we just filter the time period where we want to chart. Then, we insulate a year from dates and create a month-day column that we call a plain date. And, finally, we pivot data to have one column per year. What we do is going from raw data to a clean data set that can work in Datawrapper. So the next step is bringing that clean data into our chart template that Paula explained before. And it is really easy, thanks to MunichRocker's DatawrappR package for R, that, I'm sure, you have heard about before.

We just need four functions to modify our plot, through the API and to have it online. Let's go to them.

First of all, we copy our template. It's called "copy," the function. So it's really easy to identify. And we only need the chart id. So it's really simple.

Then we pass the clean data to the new chart that contains all the customization we did in our template, so we don't have to, edit it after the uploading the data.

In the third step, we change title, annotations. With this function, you can modify many other things. But for our chart, that's enough. And, finally we publish and recover the embed code that appears directly in our console. You just copy and send it to the team.

To sum up, once it's done, the process is very straightforward, and you can do it again and again.

You open RStudio, you connect with a data source directly, run your ready-to-go script to create the chart using your template, save it, and publish it. With this workflow, we not only can sync our timing with the breaking news team but we are even faster than them sometimes And this way RTVE can publish last-minute information already with high-quality charts. We can publish multiple charts in minutes now, but our work doesn't end here.

Examples

We always reevaluate and keep improving the process. For instance, we ask the breaking news team if the charts are still interesting for them, or if they have identified any new needs. We regularly send reminders of all charts we can create to the newsroom because they forget they have charts to use. And it's really helpful for them to have a refresher on that.

We try to include everything we learn to improve our process. And I know, it's something we'll definitely do after this instructive conference. I'm sure for that, that we are learning a lot and that we are using it. And that's broadly how we work. So thank you very much, for your time.

And, now it's time for question and answers I think.

Q: Enforcing charts (folder) structure?

Host: Thanks so much for that, really interesting stuff. My first question for you is, you mentioned having a kind of big kind of database, a big folders full of charts.How do you organize those and how do you enforce organization of those?

We have a lot of conversations and we keep reminded that we are doing this job and that this look how great it turns out when we can go with this piece and not only like a couple of paragraphs. There will be an amplification, but you go with a few graphics in the beginning. So maybe enforce is a strong word. We try to convince.

We, talk a lot about, how to organize our folder system. And it is, something that we use a lot of time on that. And it's not closed, it's changing, as Paula said in the presentation. It depends on the, news and on what's happening. Because sometimes topics change from one area to the other. It's a live system.

Q: Time it takes to make templates?

Host: Cool. All right. We've got some questions from comments. Let's see. Laura asks, how long does it take to make all those templates?

We don't do them all at once. We keep improving, like we say, our system. So we started with some very common, very obvious choices, like unemployment, or some other statistics that we get regularly, like even monthly. So we would start with that. And we would create first the template. Then we would go okay, this maybe makes sense to have in code. And then we will go to that step further. I would say we keep growing on this when we have the time. We know it will save time eventually, but we don't always have the time to invest on making all these steps, preparing for that.

When we suddenly are like "Oh, hey, there's some time here. Maybe you can prepare these for the next time", which will be, I don't know, in three months. And then we do that. We have done that in the second screen somehow. So it is not like a priority per se, because we already have the basis, but we keep working on it, and that's how we've been doing it. We started with the most important and the most obvious choices, and then, step by step, we keep going.

Yes, as Paula said in the presentation, normally, we detect the need after doing the first chart. When you have done it once, you just need to adjust that chart to be really, really polished. And, then, you use it. So it's something that is growing with our work, really. It's not a task that we have to schedule .

Q: Dealing with units?

Host: So I have another question here from Andres: How do you deal with units and date formats that us Hispanic speakers use?

Yes. That's a headache, normally . Yes. R Studio is really helpful for that because you can change the codification system really quite easily. So when you read data, you can just indentify, with, UNICODE or which. For example, in Spain we use UTF-8. So if you identify that it isn't in that code classification, you just write that in your code and it changed the data. So it's pretty much like that.

Q: Team size & roles?

Host: Great. I have a question here from Yanika next. How large is your data team and how's the team split up by roles?

We are a quite big data team. We have eight members. And we, maybe not by roles... we are all journalists and we are all growing in our data abilities. So somehow we are divided more by areas. Like one of is more specific into economics topics, or they're into more social issues or national politics.

So it will usually go like that. Like after some time we've had our beat, somehow, in the data team. And then we have one person who is more technical, and he's more in charge of maps and some more coding part. But we are very proud to say that we are all learning basic coding to more advanced coding. It will depend. But we are growing and levelling up our coding level. And we starting to use, all of us, more advanced tools to deal with the data. So I don't know, I would say we're interchangeable in many tasks.

Host: Very interesting. Okay, I think that's all the time we have for a Q& A, but thank you so much both for being here and for the presentation.

Thank you.