Surviving a Site Migration [An SEO's Guide]

Regardless of the size of your website or the revenue it drives, launching a freshly redesigned site can be a difficult and nerve-wracking time for site owners and their SEO Consultants alike. Here are some of my tips for handling a site migration well.

dice

Image credit: pasukaru76

Define and communicate your keyword strategy

They say be prepared or be prepared to fail, and your preparation for migration should start early. Keyword strategy is an oft-overlooked part of migration planning and can sometimes be left  until your site is going live. Your keyword research methodology is critical to designing a strategy that will enhance and not decimate your search traffic during the transition to live process.

Generally a site redesign, prior to launch will receive an unusual amount of interest from colleagues inside the your company. Why not use them to spot those nagging problems with your SEO? If you can communicate a keyword strategy to the company in a precise but easy to interpret manner, you’ll have a much better chance of catching any incorrect page titles, misaligned internal link anchors or sketchy meta titles.

Try documenting and diagrammatising an SEO keyword strategy, a little like this (very) simple diagram I presented at SES London 2010.

Keyword Strategy Diagram

Including details covering page titles, general keyword targeting, and an outline of your internal link strategy could yield some interesting feedback from those in your organisation who are interested in playing the SEO game.

Understand your old URL structure and write a 301 redirection map for your developers

Ideally your 301 redirection strategy should have been included in the SEO section of the functional specification of your new website months ago. If redirection has been overlooked, now might be the time to map your redirections for the development team.

If your URL structure is changing drastically, you’ll need to outline the general patterns for URL construction at varying levels of content hierarchy. I use an Excel spreadsheet with a pattern list of URL types mapped to an updated version of the same URL:

/[PRODUCTID]/[COLOUR]/?id=[X] should redirect to /[COLOUR]-[ID]-[PRODUCT]/

Your developers will be able to construct pattern matched based redirects to catch most, if not all of those dreaded 404 errors.

Use Xenu to get a list of all of your URLs before you move

Crawl your site with Xenu and export the URL list ready for post live testing. First, crawl your site with these settings:

Xenu crawler options

Next, export the URL list to a TAB separated file, extract the URL list and re-import the list into Xenu using the “Check URL list” option. Just be sure to set “crawl depth” to 0 so the crawler won’t go off and discover new links. Uncheck “treat redirections as errors” – a redirection is a success! If you come across any new errors, you’ll have a fresh list to work though. No site launch is perfect, so don’t be disappointed if you come across the odd 404 here and there.

Set up a frequent search engine visibility report

Advanced Web Ranking has an excellent visibility scoring system, expressing your overall search engine visibility as a percentage. While chasing individual rankings might be of debatable value for some, understanding the health of your site by monitoring a collection of your most important keywords can work extremely well. Consider grouping keywords from various levels of your site architecture and running checks more frequently than you would normally.

Search engine visibility - SEOgadget.co.uk

SEOgadget.co.uk visibility, top keywords, Q1 2010 – last week was awesome!

Be aware of your most valuable links

Get a report of your most valuable inbound links using your tool of choice. Whatever happens you need to make sure that none of the linked to URLs on your new site are left to error for too long, because your site won’t get the link juice it so richly deserves.

Review your best links

Changing your URL structure? Get a contacts list for your best links and realign them

If you’re planning on changing your URL structure, get a list of all of the contacts you need to contact site owners to let them know there’s been a change. After all, a 301 redirect doesn’t pass the full value of direct link.

Watch Webmaster Tools like a hawk

Webmaster tools is about the best tool we have to tell us if there’s a problem with Googlebot’s crawl of a site. Sadly the data is far from real time, but the “Crawl errors” report is an important place to start with your SEO healthchecks.

Other checks to be aware of

This is far from an exhaustive list but I hope it covers off the main issues to be aware of during a site migration. Additional activities could include checking your analytics and javascript are in place and even updating your email footers. I also think it’s extremely important to stay on top of your long tail, and do regular checks on overall site indexation to make sure that old content hasn’t been orphaned. Pages that lose all their internal links during a site migration won’t rank for long!

What tips, resources and experiences can you share the last time you launched a new website?



Stay Updated: Sign Up for Webinar & New Blog Alerts

37 thoughts on “Surviving a Site Migration [An SEO's Guide]

  1. Jordan says:

    Something every site should have anyway, but I would add make sure you have a helpful custom 404 page so that if people do find a dead link they don’t just see a generic 404 error screen.

  2. Jordan says:

    I forgot to say great article. I’m currently handling this for a client and I’m doing very similar things.

  3. David Tutin says:

    great article, like Jordan I’m currently in the process of site migration from a many to one perspective, what would we do without Xenu ehh!

  4. Thanks Jordan and excellent tip – thanks for contributing!

  5. Good luck David – the first night after a migration is usually pretty sleepless. Did I leave that out of the post? :-)

  6. Usman says:

    Hi Richard,

    Been reading your blog for a while and enjoyed reading the tips and analysis you have provided, so keep up the good work!

    This is a timely article because I’m in the process of a site migration as well. In fact we have decided to redesign the site since last summer and yet we are no where near completion!

    Like the step by step guide, in particular, the Xenu link tool which is a tool I’ve been using since the first day I started learning SEO and well worth using especially on large sites in which the site migration is the most problematic area given the taxonomy of the site – its huge!. So mapping pages using 301 is the most challening part and am glad that you have written the article, just ensures I’m on the right track!

  7. Hey Usman, thanks for getting in touch

    Good luck with the site migration – let us know how it goes!

  8. Barry Adams says:

    Good stuff Richard, it complements nicely what I wrote on the topic a while ago (http://www.greatwebsitesblog.com/2009/05/seo-concerns-when-migrating-your-website/).

  9. Excellent check list, I’ll only add to run a “site:www.yourdomain.com” in Google to check for indexed pages and/or documents.

  10. Yes – really good point, particularly the indexed documents. A simple: “site:x.com filetype:pdf” query (or docx, doc, etc) would give you plenty to check through.

  11. Matthew Brookes says:

    Nice article – a lot of people don’t think about whats involved in a site migration and then wonder why the site does not rank with the wonderful new look and feel.

    Something else to bear in mind (this was actually the first post i read on this blog) is make sure you don’t leak your staging / dev site URLS into Google etc. Otherwise you could be losing customers and creating duplicate content.

    Another point would be remember the time it can take for DNS to update so don’t just switch off the old site if moving to a completely new hosting partner.

  12. Haha – I mentioned that point on SEOmoz the other day, too. Thanks for the reminder, Matthew :-)

  13. Nate Wood says:

    One thing that I would add here is that often a simple redesign can upset your internal linking architecture, and potentially your content order in each page. I typically look at the cached version in Google and ask designers/developers to try to mirror the content order. I’ll also take a look at what’s being linked to from the homepage and ensure that the new design accounts for that link structure as much as possible. Nothing worse than getting a lovely redirect map in place only to find out that an entire section is now only linked to from deeper pages, and not the homepage any more!

  14. Hi Nate,

    Absolutely. I get a lot of SEO’s asking me to take a quick look at their new site before it goes live and a continually reoccuring theme is the number of internal links from the homepage (and from internal content blocks) has changed. You can quickly do a check for the important changes with the SEOmoz Toolbar. Its analyse page feature counts the internal links on the page. Handy.

    Hope all is well in the new job!

  15. Great post richard,
    I’m also looking at an enterprise level migration project, and so often people forget these things!
    URI structure -> everything should be mapped as to where it is currently & structure, and where it will go (via 301) and into what structure.
    Content -> Seems people forget things like PDF’s, videos and images – those are indexed to! So if your migrating dont forget them (and optimize them in the process)
    Sitemaps -> Never an afterthought!

  16. Hi Rob,

    For sure – Video seems to be cropping up a lot at the moment. Certainly with non html documents it seems something that a lot of people forget. I can’t recall a time when I’ve seen a broken PDF URL cause a 404 error report in Webmaster Tools come to think of it. Interesting…

  17. Jordan says:

    We’re actually moving from static HTML pages to ASP.NET pages. We’re currently looking at URL rewriting the .aspx file extension to .htm to keep the exact same file names where possible since 301s don’t preserve 100% link juice.

  18. That’s a really interesting idea Jordan – and given the options, it’s a choioce I’d be extremely tempted to make too. I’m pretty sure the ISAPI engine had that as one of the default rewrite options at one point. Given what we have had confirmed by Matt Cutts recently, I’d totally go for that option if it’s possible.

  19. Jordan says:

    I’m actually glad to have someone else in the field confirm that – something was niggling at me that it might not be the best idea since there doesn’t seem to be much online about it! :D

  20. Paul Gailey says:

    spoofing the suffix is fine if you are changing platforms, infact its the best thing.

    I also found that 404 referral tracking is preferable with google analytics because if done right onto your custom 404 page you can view the providence of the page /link that is producing it.indeed configure analytics to auto email instants when this happens. (Full setup details are on a google help page)

  21. Hi Paul,

    Nice comment on the 404 tracking – here’s the link you’re referring to:

    http://www.google.co.uk/support/googleanalytics/bin/answer.py?hl=en-uk&answer=86927

  22. Nate Wood says:

    I would just make sure that all links to that page by the CMS also refer to the rewritten URLs. Otherwise you’ll have the odd 301 requirement which is counterproductive to what you want to achieve. Navigation is usually the main culprit here, and I’ve seen a case where the sitewide navigation driven by the CMS would only link to the URL it had stored. Doh!

  23. Adam says:

    I simply have a header and footer for each page. Our site migrations are simply a look overhaul, which is fortunate. That way, all we need to do is change the CSS and code in the header and footer, and that’s basically it. However, for more advanced solutions, this was very very helpful. Thanks!

  24. Another EXCELLENT post, Richard! I adore your educational style! Actually, this is supreme, along with insightful comments.

    I’m wondering if you can elaborate on Xenu’s options and your suggesting setting parallel threads too 1.

    Wanted to add a step I’ve incorporated into redesign projects – a mandatory SEO Log to document each change.

  25. Hi Dana – thanks for the feedback! Always so kind :-)

    Yep – I’ll write a separate Xenu post (it’s now on my list). Love the SEO log – a bugs / issues log for SEO problems. I’ve always added a severity / impact level to mine to help the dev team decide what do do first.

    Speak soon I’m sure!

  26. Richard,
    Nice Work!
    Going to pass this on to our everyone right now and to make this SOP (standard operating procedure)

    Thank you

  27. Seobelle says:

    Great post of what to do Xenu is a great tip! Almost every site that gets migrated falls into some trouble whether its just overlooking the SEO implications from the design and dev team or pages get forgotten about.
    Good tip from Paul Gailey on tracking the 404’s

  28. vijay says:

    Another EXCELLENT post, Richard! I adore your educational style! Actually, this is supreme, along with insightful comments.

  29. Jordan says:

    Back in the days before I was an SEO and I was just finding my feet as a junior in-house web designer, we had an agency re-develop an old subscription-based tool. We moved from HTML and PHP files to .NET and they didn’t implement or even recommend or discuss 301 redirects. We dropped out of the top ten in the SERPs for our key terms and never recovered as by the time we discovered why and got them to implement 301s the damage had been done. Although the blame probably lies 50/50, I still feel to this day that such a big (and expensive!) agency should have known how to handle the switch-over better.

  30. I’ve been using Xenu for detecting 404 errors on my site, and it works great. Thanks for pointing out yet another good way to use it. And, just in time for an impending website conversion.

  31. Richard, this is an amazing article. Thanks a lot for putting it together! How often would you recommend checking keyword rankings? For most of my clients I check once a month. Would you recommend to check more often? Alex.

  32. Hi Alex,

    Well, it depends. I think most folk are comfortable with once a day for the immediate period after the migration. That said, I’m working on a car loans site migration at the moment. Owing to the competitive nature of the market, I’m watching twice a day for the first week.

    Hope that helps, and good luck!

  33. Vedette says:

    Thanks for this share, this is exactly what I was looking for.

  34. Jack Freeman says:

    Information nicely presented and useful.

  35. g1smd says:

    Make sure your Disallow All robots.txt file found on the dev or test site isn’t migrated to the live site!

    Make sure that your Google and Bing WebmasterTools verification files are re-uploaded to the live site.

    When constructing redirects from old to new URLs make sure that no request results in an unwanted multiple step redirection chain. Assuming the new site uses www URLs, especially check this for non-www requests for old URLs and for requests for a named index page and so on.

  36. Lovett says:

    I’ve found your article extremely useful as there doesn’t seem to be a great deal of resources about site migrations. Mapping URLs seems to be a real pain and it would be great if you could share ideas on how it could be automated (if possible).

  37. Toby says:

    Well, I would definitely NOT recommend this. My recommendation is: Make sure all internal links, titles, meta tags stay exactly the same. If some urls wont exist after the relaunch, just download the old pages and put them on the webserver.

    Redirects will break everything – also internal link changes….

Comments are closed.