Technical

SiteBulb. Testing the industry’s latest site crawl tool

by on 5th September 2017

This is cool.

When you open Sitebulb for the first time, you know exactly what this little tool is here to do.

Sitebulb dashboard

The main (sample) dashboard from Sitebulb. Immediately intuitive. Looks cool, too.

Report names like: “Site Speed”, “Mobile Rendering” and “Indexation Status by Depth” really stand out, as do the example charts in the sample report.

This is all stuff I might have attempted to hack together in an Excel spreadsheet a decade ago but all that effort is hardly necessary now. It’s available in this exciting new desktop crawler from the team behind URLprofiler.

Getting Started

Getting started on a new project is intuitive and easy. Even little touches like tabbing to the next form field are in place.

Sitebulb project setup

Project setup is really easy – someone’s thought about UX!

You select the reports you’d like, followed by adding the site’s XML sitemap and connecting to Google Analytics. The Google Analytics connection took some time, so for this audit I cancelled GA integration. Not that this is Sitebulb’s fault of course – Google’s APIs can be notoriously unreliable.

The crawler lets you follow its progress as it crawls your site for the first time, showing URLs crawled, various errors, 200 responses and so on. Some real thought has gone into this design, or, none at all and they’ve accidentally built a great UI. Patrick?

 Sitebulb crawl-progress

Monitor the crawler’s progress as it traverses your site for the first time.

Once the initial crawl has finished, the data analysis begins, and there are some really interesting initial insights to be had. For example, this chart shows crawled URLs by depth. Take a look at the interesting bottleneck at level 2.

Dip-in-crawl-depth-bottleneck

This website is ecommerce, so for the most part we’re talking about category level links. It might be worth investigating ways to increase the number of internal links at that level, in case we’re strangling the equity passed to the lower levels of content.

There’s a list of insights available on the Internal URLs report. All very much presented as SEO issues, in fact it’s plain to see these reports have been constructed with an SEO Audit in mind.

Internal-url-results

Sitebulb’s Internal URLs report

Issues identified on our test site include hundreds of blank, indexable date archive pages, URLs that resolve under http and https, URLs containing a double slash and more. So even in this free beta test I’ve found a load of stuff to fix while I’m writing my blog post. Thanks, guys!

Sitebulb’s crawler is a really curious little fellow and strives to leave no stone unturned.

I found a huge issue with our test site as rel="next" was incorrectly declaring a string of URLs that added a sequence of numbered URLs starting at the home page, adding thousands of blanks/duplicates. That’s wasteful crawling indeed and something I hadn’t noticed.

The Site Speed section is just really handy and to the point.

Sitebulb site-speed-performance

Site Speed reporting is thorough and very helpful, especially navigating to the problem URLs

A simple list of problem URLs in the URL hints tab means you can work through the issues with relative ease.

Slow-ttfb

Work needed on the TTFB problem – our ecommerce sub category pages are extremely database intensive.

The overview gives my site a promising 90% site speed score, but unfortunately, this eCommerce site still has some database performance issues making the TTFB (time to first byte) more than a little slow in certain content sections.

Should I get Sitebulb?

Yep. I’ve recommended it to our SEO Team as a tool you’d want to use as part of a drive-by site audit and something that you’d perhaps want to run on a monthly basis to monitor a website’s health.

This tool isn’t a replacement for crawlers like Screaming Frog or products like Ryte – it’s something you’d want in addition to them – but its unique approach to auditing, familiar lexicon and powerful little site crawler are highly complementary to the site audit process.

Responses

  1. Thanks Richard, I have been trying out URLprofiler and this seems to be a good tool from the same team. I think Internal Link analyzer & Mobile Rendering will be interesting features to watch out for. Just one question, does it help in finding broken links?

  2. What do you see as the main things this tool has that Screaming Frog doesn’t, and vice versa?

  3. Freakin’ sweet tool. Found loads of little technical things to improve on one of my sites.

    Already this good in a beta!

  4. Yes Hari it does

Your comment

Join the Inner Circle

Stay one step ahead of the competition by staying with us. Join the Inner Circle to receive a monthly update of Organic Digital marketing resources, blogs and industry news.

Get insights straight to your inbox

Stay one step ahead of the competition with our monthly Inner Circle email full of resources, industry developments and opinions from around the web.