Technical

Gathering Data with the
Sina Weibo API

by on 5th August 2014

Hong Kong. Being on the fringe of so many different cultures creates a multitude of marketing challenges that you don’t have to consider when working in the UK and the US. Being outside of what most of us would consider our comfort zones to be, it’s something I’m finding incredibly exciting to learn about.

So what happens when the majority of your audience don’t use Google to search for products, Twitter to share content, or Facebook to connect with friends? Well, the principles remain the same, but the medium with which you connect those principles to the customer changes radically.

At Builtvisible, we’ve worked hard to improve our research capability to dramatically increase success with creative and content marketing. You can read about that here, and here. But it got me thinking – could we translate the process to work when the biggest social platform in this region is Sina Weibo? To give you an idea of scale, Weibo has reported 143.8 million monthly active users which puts it at just over half the size of Twitter with 271 million monthly active users. So it’s definitely a big deal!

Thankfully Weibo offer a free API which, once you’ve got your head around the application process, is very quick to get going with. There’s a more advanced API access that requires a more in-depth application process, so for this post we’ll look at getting started with the basics.

Getting API access

To get started with making some basic calls, you’ll need to acquire the following items:

  • A Weibo Account
  • An Open Weibo Account (a developer account)
  • An App Key
  • An API secret Key
  • A Redirect URI
  • An Access Token

In terms of getting yourself set up and going through the application process, there’s a really handy step by step guide produced by the Carnegie Mellon University. They’ve done a great job of running through this so I won’t repeat everything here, but if you achieve the steps above you’ll end up in the right place.

If like myself you’re completing this process in a browser, you’ll need to make sure you’ve got something that can handle a POST request for the very last stage. Essentially, to get hold of the Access Token you need to call the Oauth2 service of the API which will only work as a POST request.

To get the job done I installed the Postman plugin for Chrome (thanks to the ever brilliant Pete Wailes for the tip). Not only does it make playing around with API calls very easy to manage, it also allows you to choose between running a GET request or POST request, meaning we can get the last stage of the application process completed easily:

Using Postman for Chrome

The parameters you’ll need for this call are:

  • client_id – that’s your app key
  • client_secret – your secret key
  • grant_type – set this to ‘authorization_code’

So, your completed API call would end up looking like:

http://open.weibo.com/wiki/2/oauth2/access_token/en?client_id=[replace with app key]&client_secret=[replace with secret key]&grant_type=authorization_code

What insights can you gain?

The staples of our social research have always revolved around identifying influencers and discovering who you want to be talking about your content, identifying hot topics within your niche and understanding your customers’ needs, and working out what your audience is sharing and therefore where you need your content placed.

With just the basic Weibo API access, not everything above is possible but you can certainly make a very good start. With the full access things should get a lot closer to what you’d be used to with your traditional social tools, but for now let’s take a look at what’s immediately available.

I want to find the most influential profiles within my industry

No problem! To do this, you can make use of the suggested users service. This function enables you to call the API and get a list of the most powerful profiles related to your keyword of choice.

Parameters needed for this are:

  • client_id – your app key
  • access_id – your access token
  • q – your keyword
  • count – the number of desired results

As such, your API call would end up looking a bit like:

https://api.weibo.com/2/search/suggestions/users.json?client_id=[app key]&access_token=[access token]&q=travel&count=100

The standard response is a JSON array, which makes things very easy when you want to import the data into Excel or Google Docs (more on that in a minute):

Weibo API JSON response

 

How can I find hot topics?

Easy. If you want to pull in a list of hot topics, you can do so by using the trends service and see topics by hour, day or week. Again, it’s a case of forming an API call with a specific set of parameters:

  • client_id
  • access_token

So, your call will look like the following, with ‘weekly’ set to ‘daily’ for topics by day, and ‘hourly’ for topics by hour:

https://api.weibo.com/2/trends/weekly.json?client_id=[app key]&access_token=[access token]

How can I measure the success of my posts?

Just like when we’re running a campaign on Twitter, you’ll probably be quite interested in keeping track of the number of reposts your updates achieve. The ‘repost timeline’ service returns a number of useful data points including the number of reposts, a name list of people who have reposted, and the follower count of everyone who’s reposted your content.

The parameters you’ll need are:

  • Client_id
  • Access_token
  • Id – the original post ID
  • Count – the maximum number of people you’d like in the list

Keeping track of it all in Google Docs

Obviously, this only starts to get useful if we have a reliable way of pulling the data from the JSON response, and creating something we can start to gain insight from. To make your life a little easier, I’ve put together a Google Docs sheet that will take care of the above three calls for you. All you need to do is make a copy of this sheet, and run through the API application process at the beginning of the post to gain the required credentials. Once you’ve got them, pop them into the first tab and away you go!

JSON in Google Docs

What’s really cool about doing this in Google Docs is that you can add a few more functions to make even more sense of what it is you’re reading. For example, in the trends tab I’ve used the =googletranslate function to translate the topic from Chinese to English:

=GOOGLETRANSLATE(A4,”cn”,”en”)

Gdocs2

Getting JSON into GoogleDocs

One of the things that made this slightly more tricky is that most of the responses are formed as a JSON array. In the past I’ve always used =JSONPathOnURL from SEOtools for Excel, but sadly it seems to trip up on arrays. The quickest workaround was to use a small script in Google Docs, allowing you to pull JSON in from a URL via the =ImportJSON function:

=importJSON(URL,Property)

Using the repost call as an example, we’re looking to pull in the screen name of each person that’s reposted your original post:

In order to do this, you need to fill out the correct path, and add that to the function. In this case, ‘screen_name’ sits within ‘user’, which sits within ‘reposts’. So, the complete function would look like this:

=ImportJSON([URL],”/reposts/user/screen_name”)

A huge thank you to Paul Gambill for an excellent description of how to get this up and running. I’d highly recommend giving it a read if you’d like to experiment more with JSON in Google Docs.

So, there you have it – although the market in Asia is very different, adjusting some of the more common methods used at home is more than possible. If you fancy having a play around, I’d love to hear what you’ve managed to pull together!

Thanks for reading, I hope you’ve found it useful, and please do drop me any questions in the comments.

Useful Resources and tools I’ve used along the way:

Join the Inner Circle

Stay one step ahead of the competition by staying with us. Join the Inner Circle to receive a monthly update of Organic Digital marketing resources, blogs and industry news.

Get insights straight to your inbox

Stay one step ahead of the competition with our monthly Inner Circle email full of resources, industry developments and opinions from around the web.