Technical

How to undertake large-scale competitive link analysis

by on 12th June 2017

Large-scale link analysis, as a process, seems to have waned in popularity over the last few years, but when executed correctly can still have real utility. Notable benefits include:

  • Helping to highlight new link opportunities, including those that may not require a content-led approach.
  • The development of a deeper understanding of the link acquisition strategies that work within a specific vertical.
  • Identification of industry specific media sites or journalists that can later be targeted by a content focused campaign.
  • More context on why specific sites are performing well (or poorly) within organic search.

Both Majestic and Ahrefs have inbuilt tools to help perform this type of analysis — Clique Hunter and Link Intersect. These allow you to enter up to 10 domains and download a spreadsheet of all of the sites that link to these domains.

Unfortunately, 10 sites often isn’t enough to capture the full competitor landscape. Instead we’re going to replicate the same functionality using Excel and a few VBA scripts.

Competitive Link Analysis In Excel

As an initial step, you’ll want to identify your key competitors and download link data from your preferred tool. I’ve used .xls exports from Majestic and 30 different recommended agencies for SEO and Content Marketing. One link per domain is usually sufficient and will help keep your dataset manageable.

Next, move these workbooks into a new folder and open up a separate blank workbook. Load up the VBA editor (ALT + F11) and click Insert > Module. Paste in the following code, making sure you update the ‘Path’ with your own folder.

Click the green play button to run or press F5.

Competitor Link Analysis 1

Let Excel chug away for a while and when it’s finished you should have a workbook with one domain per sheet.

Right click on the worksheet bar and select all sheets. On the sheet you’re already viewing, select column E (domain), then cut and paste it into column A. This should perform the action across all of the sheets.

Go back to the VBA editor, insert a new module and add the following:

Run the script which should create a new sheet at the beginning called ‘combined’.

View the sheet and then go to Data > Remove Duplicates. Only tick ‘Domains’. This will trim the dataset down so that only one example URL remains per domain; usually this is sufficient.

Competitor Link Analysis 2

Add in a new column to check whether or not the domains already link to your site, using the following formula:

Reference the domain column on your combined sheet and then the domains column of the worksheet for your site.

Finally, we’re going to check how many competitors each domain on the combined sheet links to. Create a new VBA module and paste in the following:

Go back to the combined sheet, insert a new column and use the following formula, referencing column A of your first non-combined sheet again. The VBA will perform a count across all of the sheets except for the first (which should be your combined one).

Format the data to your liking and you’re ready to go!

Competitor Link Analysis 3

FAQs

How many sites can this handle?

Assuming you have a decent machine, this should work fine with 30~ sites with reasonably large link profiles (~5K domains). Limiting it to one link per domain helps. Beyond this, it may be best to use SQL, R or Python.

Can I use exports in different formats?

Absolutely. The initial VBA could easily be tweaked to work with CSV’s instead.

Can I use a mixture of link metrics?

If you want to use link data from Majestic and then overlay a metric like DA, then just run the combined domains list through a tool like URL Profiler and pull the data into the sheet with a VLOOKUP.

Responses

  1. This is a very helpful post Will! :) I was wondering what other tools aside from Majestic and Ahrefs we can use for this?

Comments are closed.

Join the Inner Circle

Stay one step ahead of the competition by staying with us. Join the Inner Circle to receive a monthly update of Organic Digital marketing resources, blogs and industry news.

Get insights straight to your inbox

Stay one step ahead of the competition with our monthly Inner Circle email full of resources, industry developments and opinions from around the web.