Quick ways to find and manage out of stock products at scale

Finding out of stock products

You can use the ‘Custom search’ feature in Screaming Frog to find out of stock products or empty categories if they return a 200 status code (as these won’t be picked up via a standard crawl or Google Search Console’s 404 report):

Step 1: Find the identifiable out of stock copy on your product page. Using Amazon as an example, their identifiable out of stock is “currently unavailable.”:

Step 2: Copy and paste this into Screaming Frog’s ‘Custom Search’ feature (make sure you copy from the HTML of the page, as this is where Screaming Frog will look at it from):

Bear in mind that the search function is case sensitive:

Step 3: Run a crawl of the site using that custom search to return all of the product pages that contain the “out of stock” string and don’t forget to manually QA for any errors:

You can use the same process to find product listing pages that are empty (meaning they have no products). Here are some examples:

Just copy the ‘no products’ identifier in Screaming Frog, in the same way we did for ‘out of stock’ products.

Finding thin category pages

To find thin category pages with limited number of products you can use Screaming Frog’s ‘Custom extraction’ tool:

Taking this ASOS category page as an example:

You can use the tool to fetch the contents of any container using class=“styleCount” by typing the below:

//*[contains(@class, 'styleCount')]

If your site is too big and you are having issues with allocated memory on your desktop, you can limit your crawl to only include category URLs or exclude product URLs by using the ‘include’ or ‘exclude’ features  on the tool:

Once the crawl is finished, go to the ‘Custom’ tab at the top of the tool and you’ll see something like this:

In some cases, if your page doesn’t have a container with the number of products available, like this example here, you can still count the number of elements on a page by using the count function instead of the search one:

count(//div[@class="offer__content"])

If you want to learn more about how to use XPath for SEO purposes, you can read this guide. To extract other elements of your category pages, such as titles, headings, etc. you can look at this article.

Managing out of stock products at a large scale

Handling out of stock products (both permanently and temporarily unavailable) at scale can be very time consuming. On many occasions, you might not know whether a product is coming back in stock or how long it’ll take to arrive. For these reasons, I recommend setting up some rules on the server that will automatically take care of out of stock product pages for you.

In order to avoid Panda-related penalties and issues with thin content, as part of this process you’ll also need to deal with category pages with a limited product offering (say anything with less than 3-4). A category page with only 2 or 3 products will rarely fulfil a user’s intent and is therefore unlikely to rank. Even if it does, it is significantly less likely to drive conversions and revenue to the site, so there might not be any real benefit in these being indexed.

For this purpose, you can set up dynamic indexing by implementing some rules. For example, after 30 days of a certain product being out of stock or a product listing page being empty (or thin), you should set them to noindex. When the product is back in stock or the product listing page has enough product offering again, the noindex tag should be removed.

If, after 30 more days (so 60 in total), the product is not available or the category page is still empty it probably means the products have been discontinued and can be treated as explained in the first part of this series i.e. with a 301 redirect to one of the following (in order of priority):

1. A similar product / category
2. Its parent category
3. Homepage (only if none of the above is possible)

Note that this time stamp has worked for most of our clients but might not be suitable for your site (e.g. sometimes products take more than 30 days to come back, in which case, the time stamp should be extended accordingly).

Also, remember that the noindex tag recommendation doesn’t apply to highly competitive terms (e.g. ‘Christmas’ or ‘Black Friday’), which will fall into the first group mentioned here.

The only problem with this is that, over time, you’ll be accumulating a lot of stuff in the .htaccess file, so it’s important that you clean your old legacy redirects once they’re not needed anymore. You can read how to do this here.

And that’s that!

A correctly implemented product retirement strategy will help you avoid indexation bloat and waste of crawl budget, which will ultimately benefit your site’s overall organic performance. But remember that you’ll also need to keep your users happy. Avoiding frustrations and disappointments for your users will ultimately have a positive impact on conversions (and therefore, revenue). For this purpose, make sure you notify ‘out of stock’ products as clearly and quickly as possible.

Finally, remember that every user who visits an out of stock item is a potential sale; you’ll want to make sure you recover these lost sales by suggesting a similar or related product to users and notifying them when the item is back in stock.