How to gather data to craft newsworthy stories

Take advantage of APIs

Working with existing data sets is a great way to spin new stories from reputable sources. You can find plenty of data banks online – like the World Bank or Numbeo – but some of the best data sources take more effort to mine.

Thankfully, many of the biggest and therefore most data-rich websites have APIs that you can access to request information that interests you. If you’re unfamiliar with how an API works, here’s a metaphor from How To Geek that will give you the gist:

Think of an API like a menu in a restaurant. The menu provides a list of dishes you can order, along with a description of each dish. When you specify what menu items you want, the restaurant’s kitchen does the work and provides you with some finished dishes. You don’t know exactly how the restaurant prepares that food, and you don’t really need to.

Here’s an example of how an API can be leveraged for content creation: a brief we received at Builtvisible last year was to create a campaign that generates media coverage for a brand who wants to appeal to people interested in a hiring a car.

Knowing that most people hire a car for leisure, we wanted to equate hiring a car with a going on an adventure, so we set out to discover what music people choose to soundtrack their road trips.

Once we started looking into data sources, Spotify quickly emerged as a natural place to look for this project. Like just about all popular apps, Spotify has its own web API, so we collated a long list of user-generated playlists related to ‘road trips’ or ‘driving’. We then requested the contents of those playlists from the web API, along with some other metrics that we were interested in such as the popularity and length of each individual song. This gave us a huge data set to analyse – once the legwork was done, the story was easy to piece together.

Plenty of other websites have APIs that you can access. The data wizards at The Pudding collected tons of information from Wikipedia’s API to track how often certain celebrities are mentioned in the media. They also used a similar approach to plot 100 years of news headlines from the New York Times’ headlines API archive.

Extract powerful data from other websites

You shouldn’t underestimate the power of ‘scraping’ websites either – you don’t need to be a developer to do this, because there are plenty of free tools that do the hard work for you. Let’s say you want to find out where the best Indian restaurants in London are located according to TripAdvisor.

The long way to do this would be to open each restaurant page from TripAdvisor’s rankings and jot down the address of each restaurant. Here’s how to extract information from multiple webpages in five minutes:

1. Gather the URL for every webpage that you want to include in your data set: There are various free browser extensions you can use to extract every link to a restaurant page from TripAdvisor’s rankings into a spreadsheet.
2. Identify the element on a webpage that you want to extract using Scraper for Chrome: Simply right click on the restaurant’s address and select ‘Scrape similar’. This popup will show you the XPath reference for you to copy to your clipboard.
3. Extract specific information from a list of webpages in Screaming Frog: Paste your list of URLs into Screaming Frog and use its powerful custom extraction tool to extract the restaurant address from each page by specifying the XPath reference that you copied from Scraper for Chrome.

Builtvisible’s founder Richard Baxter has written a step-by-step guide to scraping web data that goes into much more detail, so make sure you read that before you start scraping data from websites.

Mining private data

Any brand with an online presence will have plenty of data you can use to tell a story. Google Analytics can tell you everything from what browsers your customers are using to what products they are buying. This might not seem particularly newsworthy, but you can mine your private data to find interesting angles and stories.

When working with a leading online estate agent, we used GTM tagging on their property listings to identify which rooms in a house people spend the most amount of time looking at. Journalists know that this kind of data is highly useful for anyone interested in selling a property. From our findings, we could show that people spend longer looking at a living room than a kitchen or bathroom, which added an element of surprise.

Doing your own research

The third and final data source for newsworthy stories is that which doesn’t exist yet. This is your opportunity to do your own research to position your brand as an authority on a topic.

To ensure your research yields newsworthy findings, your methods must be robust. Journalists have historically regarded themselves as ‘gatekeepers’ of information. Your research must meet the same editorial standards that their editor would demand from them. Very few editors, for example, will consider a Google Survey that includes 100 respondents as robust.

Here are some tips for doing your own primary research to create newsworthy data:

• Run an online survey to generate quantitative data (pro tip: don’t rely on a survey as your only source of data, as journalists are naturally quite sceptical of this method)
• Hold a focus group
• Crowdsource opinions from an online community such as Reddit
• Conduct a social experiment and record the findings
• Interview experts or speak to people who have a story to tell
• File a Freedom of Information request – you can normally expect a response within 20 working days

Making sense of raw data

Raw data can be messy and meaningless. Turning numbers in a spreadsheet into a newsworthy story often requires a fair amount of manual processing before you can start writing your story.

Thankfully, there are brilliant websites like Excel Jet, which are invaluable reference points while you learn your way around the formulas.

Here are some tools and functions that will prove to be very handy when working with data sets in Excel:

• Pivot tables: Essential for exploring bigger data sets and highlighting key information.
• Conditional formatting: A simple way to highlight values in your data set that meet a certain criterion.
• Calculate percentage change: Useful for expressing trends and value changes over time.
• Text to columns: A life-saver when you want to turn a messy string of values into a data in a table.
• VLOOKUP: Instantly match up different data sets with the godfather of all formulas.
• COUNTIF: The easiest way to count how many times a value or string of text appears within a range.
• TRIM: A quick hack for removing errant spaces from raw data.
• CONCATENATE: A terribly named formula that allows you to join two or more strings together.

Getting to the heart of your story

If you want to tell journalistic stories in a digital age, sharpening your Excel skills and brushing up on your number-crunching can be just as important as having a good command of language.

Within this blog post, we have looked at data as a source for stories, rather than a vehicle for storytelling. In the book ‘Made to Stick,’ Chip and Dan Heath teach us that numbers are abstract, and concrete imagery and simple stories are what linger in people’s minds.

All good stories need a clear narrative and a compelling hook, whether that’s an element of surprise, a lesson that adds real value or an emotional trigger. If you have a story to tell, ensuring it’s based on hard data can only improve the chances it will be retold.