Double-checking and analysis

The team analyzed the data in two main ways: Via the database front end

This entailed typing particular keywords of interest (e.g., “tobacco,” “hotel,” “company A” into the search engine. With help of Google Translate, which was plugged into the search functionality of our database, those keywords would be translated into 21 languages and would return appropriate results. These could be downloaded and reporters could do further research on the individual projects of interest.

By macro-analysis using the whole database

Occasionally, we would download a full dataset, which could then be analyzed (for example, using keywords, or aggregating data by country, region, type of ex- penditure, number of projects by beneficiary, etc.)

Our story lines were informed by both these methods, but also through on-the-ground and desk research.

Double-checking the integrity of the data (by aggregating and checking against what authorities said had been allocated) took a substantial amount of time. One of the main problems was that authorities would for the most part only divulge the amount of “EU and national funding”. Under EU rules, each program is allowed to fund a certain percentage of the total cost using EU funding. The level of EU funding is determined, at program level, by the so-called co-financing rate. Each program (e.g., regional com- petitiveness) is made up of numerous projects. At the project levels, technically one project could receive 100 percent EU funding, and another none at all, as long as grou- ped together, the amount of EU funding at the program level is not more than the approved co-financing rate.

This meant that we needed to check each EU amount of funding we cited in our stories with the beneficiary company in question.

— Cynthia O’Murchu, Financial Times

The Eurozone Meltdown

So we’re covering the Eurozone meltdown. Every bit of it. The drama as governments clash and life savings are lost; the reaction from world leaders, austerity measures, and protests against austerity measures. Every day in the Wall Street Journal, there are charts on jobs loss, declining GDP, plunging world markets. It is incremental. It is numbing.

The Page One editors call a meeting to discuss ideas for year-end coverage and as we leave the meeting, I find myself wondering: what must it be like to be living through this?

Is this like 2008 when I was laid off and dark news was incessant? We talked about jobs and work and money every night at dinner, nearly forgetting how it might upset my daughter. And weekends, they were the worst. I tried to deny the fear that seemed to have a permanent grip at the back of my neck and the anxiety tightening my rib cage. Is this what was it like right now to be a family in Greece? In Spain?

I turned back and followed Mike Allen, the Page One editor, into his office and pitched the idea of telling the crisis through families in the Eurozone by looking first at the data, finding demographic profiles to understand what made up a family and then surfacing that along with pictures and interviews‚ audio of the generations. We’d use beautiful portraiture, the voices—and the data.

Back at my desk, I wrote a précis and drew a logo.

Figure 3-3. The Eurozone Meltdown: precis (Wall Street Journal)

For the next three weeks I chased numbers: metrics on marriage, mortality, family size, and health spending. I read up on living arrangements and divorce rates, looked at surveys on well-being and savings rates. I browsed national statistics divisions, called the UN population bureau, the IMF, Eurostat, and the OECD until I found an econo- mist who had spent his career tracking families. He led me to a scholar on family com- position. She pointed me to white papers on my topic.

With my editor, Sam Enriquez, we narrowed down the countries. We gathered a team to discuss the visual approach and which reporters could deliver words, audio and The Eurozone Meltdown | 67

story. Matt Craig, the Page One photo editor, set to work finding the shooters. Matt Murray, the Deputy Managing Editor for world coverage, sent a memo to the bureau chiefs requesting help from the reporters. (This was crucial: sign-off from the top.) But first the data. Mornings I’d export data into spreadsheets and make charts to see trends: savings shrinking, pensions disappearing, mothers returning to work, health spending, along with government debt and unemployment. Afternoons I’d look at those data in clusters, putting the countries against each other to find stories.

I did this for a week before I got lost in the weeds and started to doubt myself. Maybe this was the wrong approach. Maybe it wasn’t about countries, but it was about fathers and mothers, and children and grandparents. The data grew.

And shrank. Sometimes I spent hours gathering information only to find out that it told me, well, nothing. That I had dug up the entirely wrong set of numbers. Sometimes the data were just too old.

Figure 3-4. Judging the usefulness of a dataset can be a very time-consuming task (Sarah Slobin)

And then the data grew again as I realized I still had questions, and I didn’t understand the families.

I needed to see it, to shape it. So I made a quick series of graphics in Illustrator, and began to arrange and edit them.

Figure 3-5. Graphic visualization: making sense of trends and patterns hidden in the datasets (Sarah Slobin)

Figure 3-6. Numbers are people: the value of data lies in the individual stories they represent (Wall Street Journal)

We launched. I called each reporter. I sent them the charts, the broad pitch and an open invitation to find stories that they felt were meaningful, that would bring the crisis closer to our readers. We needed a small family in Amsterdam, and larger ones in Spain and Italy. We wanted to hear from multiple generations to see how personal history shaped responses.

From here on in, I would be up early to check my email to be mindful of the time-zone gap. The reporters came back with lovely subjects, summaries, and surprises that I hadn’t anticipated.

For photography, we knew we wanted portraits of the generations. Matt’s vision was to have his photographers follow each family member through a day in their lives. He chose visual journalists who had covered the world, covered news and even covered war. Matt wanted each shoot to end at the dinner table. Sam suggested we include the menus.

From here it was a question of waiting to see what story the photos told. Waiting to see what the families said. We designed the look of the interactive. I stole a palette from a Tintin novel, we worked through the interaction. And when it was all together and we had storyboards, we added back in some (not much but some) of the original charts. Just enough to punctuate each story, just enough to harden the themes. The data be- came a pause in the story, a way to switch gears.

In the end, the data were the people; they were the photographs and the stories. They were what was framing each narrative and driving the tension between the countries. By the time we published, right before the New Year as we were all contemplating what was on the horizon, I knew all the family members by name. I still wonder how they are now. And if this doesn’t seem like a data project, that’s fine by me. Because those moments that are documented in Life in the Eurozone‚ these stories of sitting down for a meal and talking about work and life with your family was something we were able to share with our readers. Understanding the data is what made it possible.

— Sarah Slobin, Wall Street Journal

Covering the Public Purse with OpenSpending.org

In 2007, Jonathan came to the Open Knowledge Foundation with a one page proposal for a project called Where Does My Money Go?, which aimed to make it easier for UK citizens to understand how public funds are spent. This was intended to be a proof-of- concept for a bigger project to visually represent public information, based on the pio- neering work of Otto and Marie Neurath’s Isotype Institute in the 1940s.

Figure 3-8. Where Does My Money Go? (Open Knowledge Foundation)

The Where Does My Money Go? project enabled users to explore public data from a wide variety of sources using intuitive open source tools. We won an award to help to develop a prototype of the project, and later received funding from Channel 4’s 4IP to turn this into a fully fledged web application. Information design guru David McCand- less (from Information is Beautiful; http://www.informationisbeautiful.net/) created sev- eral different views of the data that helped people relate to the big numbers—including Covering the Public Purse with OpenSpending.org | 71

the “Country and Regional Analysis,” which shows how money is disbursed in different parts of the country, and “Daily Bread”, which shows citizens a breakdown of their tax contributions per day in pounds and pence.

Figure 3-9. The Where Does My Money Go? Daily Bread tax calculator (Open Knowledge Foundation)

Around that time, the holy grail for the project was the cunningly acronymed Combined Online Information System (or COINS) data, which was the most comprehensive and detailed database of UK government finance available. Working with Lisa Evans (before she joined the Guardian Datablog team), Julian Todd and Francis Irving (now of Scra- perwiki fame), Martin Rosenbaum (BBC), and others, we filed numerous requests for the data—many of them unsuccessful (the saga is partially documented by Lisa in the sidebar “Using FOI to Understand Spending” on page 120).

When the data was finally released in mid-2010, it was widely considered a coup for transparency advocates. We were given advance access to the data to load it into our web application, and we received a significant attention from the press when this fact was made public. On the day of the release, we had dozens of journalists showing up on our IRC channel to discuss and ask about the release, as well as to enquire about how to open and explore it (the files were tens of gigabytes in size). While some pundits claimed the massive release was so complicated it was effectively obscurity through

transparency, lots of brave journalists got stuck in the data to give their readers an unprecedented picture of how public funds are spent. The Guardian live-blogged about the release and numerous other media outlets covered it, and gave analyses of findings from the data.

It wasn’t long before we started to get requests and enquiries about running similar projects in other countries around the world. Shortly after launching OffenerHaushalt —a version of the project for the German state budget created by Friedrich Lindenberg —we launched OpenSpending, an international version of the project, which aimed to help users map public spending from around the world a bit like OpenStreetMap helped them to map geographical features. We implemented new designs with help from the talented Gregor Aisch, partially based on David McCandless’s original designs.

Figure 3-10. OffenerHaushalt, the German version of Where Does My Money Go? (Open Knowledge Foundation)

With the OpenSpending project, we have worked extensively with journalists to ac- quire, represent, interpret, and present spending data to the public. OpenSpending is first and foremost an enormous, searchable database of public spending—both high- level budget information and transaction-level actual expenditure. On top of this are built a series of out-of-the-box visualizations such as treemaps and bubbletrees. Anyone can load in their local council data and produce visualizations from it.

While initially we thought there would be a greater demand for some of our more sophisticated visualizations, after speaking to news organizations we realized that there were more basic needs that needed to be satisfied first, such as the the ability to embed dynamic tables of data in their blogposts. Keen to encourage news organizations to give the public access to the data alongside their stories, we built a widget for this too. Our first big release was around the time of the first International Journalism Festival in Perugia. A group of developers, journalists and civil servants collaborated to load Italian data into the OpenSpending platform, which gave a rich view of how spending was broken down amongst central, regional, and local administrations. It was covered in Il Fatto Quotidiano, Il Post, La Stampa, Repubblica, and Wired Italia, as well as in the Guardian.

Figure 3-11. The Italian version of Where Does My Money Go? (La Stampa)

In 2011 we worked with Publish What You Fund and the Overseas Development In- stitute to map aid funding to Uganda from 2003-2006. This was new because for the first time you could see aid funding flows alongside the national budget—enabling you to see to what extent the priorities of donors aligned with the priorities of governments. There were some interesting conclusions—for example, both counter HIV programs and family planning emerged as almost entirely funded by external donors. This was covered in the Guardian.

We’ve also been working with NGOs and advocacy groups to cross-reference spending data with other sources of information. For example, Privacy International approached us with a big list of surveillance technology companies and a list of agencies attending a well-known international surveillance trade show, known colloquially as the ‘wire- tappers ball’. By systematically cross-referencing company names with spending data- sets, it was possible to identify which companies had government contracts—which could then be followed up with FOI requests. This was covered by the Guardian. We’re currently working to increase fiscal literacy among journalists and the public as part of a project called Spending Stories, which lets users link public spending data to public spending related stories to see the numbers behind the news, and the news around the numbers.

Through our work in this area, we’ve learned that:

• Journalists are often not used to working with raw data, and many don’t consider it a necessary foundation for their reporting. Sourcing stories from raw information is still a relatively new idea.

• Analyzing and understanding data is a time-intensive process, even with the nec- essary skills. Fitting this into a short-lived news cycle is hard, so data journalism is often used in longer-term, investigative projects.

• Data released by governments is often incomplete or outdated. Very often, public databases cannot be used for investigative purposes without the addition of more specific pieces of information requested through FOI.

• Advocacy groups, scholars, and researchers often have more time and resources to conduct more extensive data-driven research than journalists. It can be very fruitful to team up with them, and to work in teams.

— Lucy Chambers and Jonathan Gray, Open Knowledge Foundation

Finnish Parliamentary Elections and Campaign Funding