SiteBulb. Testing the industry’s latest site crawl tool

Getting Started

Getting started on a new project is intuitive and easy. Even little touches like tabbing to the next form field are in place.

Project setup is really easy – someone’s thought about UX!

You select the reports you’d like, followed by adding the site’s XML sitemap and connecting to Google Analytics. The Google Analytics connection took some time, so for this audit I cancelled GA integration. Not that this is Sitebulb’s fault of course – Google’s APIs can be notoriously unreliable.

The crawler lets you follow its progress as it crawls your site for the first time, showing URLs crawled, various errors, 200 responses and so on. Some real thought has gone into this design, or, none at all and they’ve accidentally built a great UI. Patrick?

Monitor the crawler’s progress as it traverses your site for the first time.

Once the initial crawl has finished, the data analysis begins, and there are some really interesting initial insights to be had. For example, this chart shows crawled URLs by depth. Take a look at the interesting bottleneck at level 2.

This website is ecommerce, so for the most part we’re talking about category level links. It might be worth investigating ways to increase the number of internal links at that level, in case we’re strangling the equity passed to the lower levels of content.

There’s a list of insights available on the Internal URLs report. All very much presented as SEO issues, in fact it’s plain to see these reports have been constructed with an SEO Audit in mind.

Sitebulb’s Internal URLs report

Issues identified on our test site include hundreds of blank, indexable date archive pages, URLs that resolve under http and https, URLs containing a double slash and more. So even in this free beta test I’ve found a load of stuff to fix while I’m writing my blog post. Thanks, guys!

Sitebulb’s crawler is a really curious little fellow and strives to leave no stone unturned.

I found a huge issue with our test site as rel="next" was incorrectly declaring a string of URLs that added a sequence of numbered URLs starting at the home page, adding thousands of blanks/duplicates. That’s wasteful crawling indeed and something I hadn’t noticed.

The Site Speed section is just really handy and to the point.

Site Speed reporting is thorough and very helpful, especially navigating to the problem URLs

A simple list of problem URLs in the URL hints tab means you can work through the issues with relative ease.

Work needed on the TTFB problem – our ecommerce sub category pages are extremely database intensive.

The overview gives my site a promising 90% site speed score, but unfortunately, this eCommerce site still has some database performance issues making the TTFB (time to first byte) more than a little slow in certain content sections.

Should I get Sitebulb?

Yep. I’ve recommended it to our SEO Team as a tool you’d want to use as part of a drive-by site audit and something that you’d perhaps want to run on a monthly basis to monitor a website’s health.

This tool isn’t a replacement for crawlers like Screaming Frog or products like Ryte – it’s something you’d want in addition to them – but its unique approach to auditing, familiar lexicon and powerful little site crawler are highly complementary to the site audit process.