Google Page Level Penalty for Comment Spam – Rankings and Traffic Drop

The morning after I arrived home from my post SMX Advanced holiday I got up early to check my site traffic. Not to mention the personal achievement of being up and working before 7am on a Monday (this is good for me…), I even managed to resist the temptation of logging into Analytics on my iPhone for an entire week while I was in Italy too. Surely the start of a very productive week. Sadly, I found a bit of a suprise. My blog traffic had dropped!

To ease myself back into blogging (and to provide some traffic dropping, Google page penalty based entertainment for SEOgadget readers) here’s the story on what happened:

Comparing the first half of June to the last half of May, overall traffic on the site had dropped by 4.52%.

week-on-week-traffic-comparison

This was pretty disappointing so I drilled down a little deeper by traffic source, showing a decrease in search engine traffic as the main culprit:

google-traffic

The great thing about Google analytics is how quickly and easily you can drill down to the specifics. Most of us use Google Analytics every day but it’s often only until something a bit unpleasant happens to your site that you really appreciate the beauty of this (free) software.

Drilling down to the keyword level I quickly found the culprit:

vbox-keywords

Those Virtualbox keywords are from the top of the tail generated by a guide to installing Virtualbox guest additions. “Virtualbox guest additions” is was  my 2nd most popular keyword generating 1500 visits in May 2009. So what was going on?

Comment spam, missed by Akismet. Don’t get me wrong, I think Akismet is amazing, but it can miss some types of comment spam. It’s probably my fault for not adding a verification or a CAPTCHA to my comments are but I don’t enjoy the experience on other blogs personally, so I choose to leave that off.

Here’s a sample of what was on the page: (The following image may offend some readers)

spam

Those links were going to some seemingly bonafide domains that happened to have some very hacked looking, spammy page URLs on them. As you can see, they were left on June 3rd, at 6.10am.

Nearly all of the traffic coming to the Virtualbox post dropped within 24hours and stayed that way until I cleared the comments on Monday 15th June at 7.00am (ish). Here’s the screenshots of the Google.co.uk results:

Before:

before penalty

After:

after

Thankfully, the page was reincluded within 24 hours of clearing out the comments. Here’s the timeline of events according to Google Analytics (and Snagit, of course.) – Click to enlarge:

traffic-small

So, the traffic has returned and SEOgadget is comment spam free once more. What did I learn from this? I thought it was interesting to see how quickly the page was dropped from the index and how quickly it ranked after the clean up. 24hours each way, no reinclusion request needed. That’s good to know. I discussed the issue with Adam from work and we concluded that it would be quite interesting to look at whether it was  the words used in the comments or the links used (nofollowed) or a combination of the two. That’s quite an easy test to do…

Just until the cached copy updates, you can see what Google saw just here.

By the way, if this is your first visit to SEOgadget, thanks! We’re an SEO Agency and content marketing provider based in London, UK. This is one of our most popular pages but we’ve got a ton of interesting articles for you to read. Thanks for stopping by!



Stay Updated: Sign Up for Webinar & New Blog Alerts

78 thoughts on “Google Page Level Penalty for Comment Spam – Rankings and Traffic Drop

  1. Erik says:

    Nice piece of work. And also great to know that your page came back in the SERP within 24 hours.

    Probably it helps that your site has a lot of readers therewith increasing the indexrank of the page.

    It also brings up the necessity to keep moderating pro actively.

  2. Nice sum up and this show how fast Google react to things happening in our pages. Happily they are fast to get your ranking as it was before when you take the time to fix the damn spam.
    Seems you had a great trip in Italy, a week off worth the little spam that get through ;)

  3. Tom says:

    Nice catch Rich – it’s very interesting to see how quickly this took effect. I’d love to know the outcome of any tests done into whether it was the link or the text. I’d suspect the text, but I could easily be wrong.

  4. robwatts says:

    Nice piece of analysis, like the way you show a clear correlation of cause and effect – tweeting…

  5. David Airey says:

    Hi Richard,

    Glad to see such a quick reversal after the comments were cleared. I think you’re right not to add a captcha for comments. Some can be notoriously difficult to bypass.

  6. @Tom – sure will. The test is running now :-)

    @David – agreed. It’s good that Google clear this type of spam penalty so rapidly. Very clever.

  7. Nice post Richard, it sure does illustrate JUST how seriously the old ‘Don’t link to bad neighbourhoods’ advice should be taken.

  8. @Old Welsh Guy

    Exactly – though the question on my mind is, what cause the filter? Is it the links (nofollowed?), the text, or both. That’s what we’re testing now – just to be sure. Thanks for your comments! Love the name, by the way!

  9. @Richard It raises the question (again) of just how ‘nofollow’ Google treats the NOFOLLOW tag. Could this ‘penalty’ (for want of a better word), be the result not so much of the adult content itself, but a simple case of non semantically related content close to links, thus dropping the Google trust?

    Ironic that Google, a search engine built on the backrub backlinking citation algorithm, appear to be doing all in their power to stop people linking out with confidence.

  10. Interesting. *whistles*

    Was there a reinclusion request submitted, or just comment scrubbing?

  11. Tonnie says:

    @richardbaxter,

    I almost for sure know it is the text that kicked the filter.

    The drop cant be caused by a nofollow on the links, Google even states you should put a nofollow on links you dont trust.

    On several domains i have seen a drop in rankings for a certain page that contained just a little piece of text one could see as related to sexual content.

  12. @Tonnie, That is a real assumption to say it is the text that caused this, as the algo is way more complex than that isn’t it.

    It could simply be that the comments were allowed to show (regardless of nofollow), and as a result Google decide the site has lost some editorial integrity and drops the trust element accordingly.

    So by that token it wouldn’t actually have been the text, it was the fact it went unmoderated that caused it. I know this is semantics, but many people (not saying you), read things differently and would take it that the use of those words caused the problem, when they didn’t, as there are a lot of sites who use those words, who rank for those words as that is what their site is about.

  13. S.Smith says:

    Great, informative post…what will you add to eliminate future spam?

    I use WP-Spamfree in addition to Akismet, because I never like Captchas or Disqus or other kinds of interventions either.

  14. Very informative Richard thanks for taking the time to put the post together. This certainly makes a case for keeping a close eye on the comments.

  15. @Old Welsh Guy

    “it was the fact it went unmoderated that caused it”

    I like that point – well made.

  16. Sasa says:

    Guys, I am using a couple of your images in my blog post. Please let me know if this is a problem and I will take them down asap.

    Very interesting. It would be very nice if you could test this. I know you probably dont wann risk too much. But it would be funny to see what happens if you reactivate these comments from time to time ;-)

    Thanks for sharing

  17. Sasa says:

    Oh yeah, maybe I should post the URL to the blog entry: Moderieren Sie Ihre Kommentare oder riskieren Sie schlechtere Rankings which translated would read something like: Moderate your comments or risk losing your rankings

  18. I’m really starting to think that I better start using paged comments. Presumably Google would start sending traffic again when the bad comments got off the main page (ie onto page 2 of the comnments)?

    Who knows – but this along with the recent PageRank sculpting changes make paged comments look better and better (with the canonical tag to prevent duplicate content penalty).

  19. Gareth James says:

    I am suprised that these effects and changes happened so quickly. I could understand if the site was PR5-6 and being crawled frequently. It seems so easy to inflict negative seo on a competitor…I may have to start up Xrumer!

  20. sylvain says:

    Really good analyse ! But when we look the last image, the visits have already started to grow up before you clear the comments ! Is it normal ?

  21. @sylvain not quite. I cleared those comments very early on the day in question. If I started getting traffic again before the end of that day, the graph will appear to increase.

  22. Tonnie says:

    @old welsh guy

    Aint that difficult, the algo has not much to do with it, just a filter that kicks in on certain words.

    That there are sites ranking high that still use those words, might be, on the other hand, there are still sites that do spam and rank too ;)

    Dont think to difficult when it comes to Google. It aint that complicated.

    Further more, if a ‘unmoderated’ penalty would have taken place, how come it did so fast?

    Wouldn’t it be more logical that Google throws one in after a certain period?

    Just one week and your out of business?

  23. Wiep says:

    I tend to agree with Tonnie here – it looks a bit like a text filter. I know it’s too late now, but I’d like to know if your website would show up with SafeSearch turned off…

    Also, the complete top 4 for ‘virtualbox guest additions’ is different in both screen shots. Could you think of an explanation for this?

  24. Really Interesting this post.

    It´s a flag to pay more attention in the comment links, mainly after the changes on nofollow rules.

  25. @Wiep Hello mate. Actually Will at Distilled said the same thing last night.

    As far as the totally different rankings go, perhaps a possible indicator of how a shorter term index may be deployed in case the main index changes unexpectedly when a penalty occurs?

  26. DennisG says:

    Really interesting stuff.
    I really hate those comment spammers, and I always make sure all posts are closed when I have no way of moderating the comments while I’m kicking it on the beach. But it makes me sad that my readers don’t have the chance of interacting with me while I’m sipping on a cocktail.

    Thanks for sharing this. Very helpful

    Dennis

  27. twitrounds says:

    Great post! We allow comments on our site and it’s good to look out for this stuff. Fortunately we don’t allow comments to be posted without approval so hopefully we never run into this problem!

  28. Nice post. Interesting, but not sure that a bunch of comment can attract the attenction of Google penalty system.
    I’ve a site of a company who I manage that seems page level penalized, and it’s a standard plain html web site, without comment, so the reason must be searched somewhere else.

  29. John Rothko says:

    I have my comments set to approval before they are published, so this sort of problem won’t happen to me. Indeed, there are actually quite a few spam comments that escape Askimet and that is not normal, automation only goes up to a point.
    Thanks for the insight. I didn’t know it would affect a site so fast.
    Keep up the good work :-)

  30. John Rothko says:

    Sorry, I meant to say that it is not abnormal that some spam escapes Askimet :-)

  31. Marie says:

    wow.. nice post. I thought akismet is close to perfection. huhuhu.. i learned a lot, comments do affect rankings. :D thank u!!

  32. John Rothko says:

    Askimet works with a list of known spam messages. In that sense it is perfect, but you cannot do anything against a person who fills in a spam message by hand.
    But it is a lot less, and I mean really a lot less then without the protection from Askimet, so I would donate some money towards those guys as they really do a good job.

  33. IBL Builder says:

    Am I alone in thinking “so what”. If you are naive enough in this day and age to allow comments onto your blog without moderation, then why would you be surprised by Google’s response when they find them?

  34. @IBL builder

    I recieve a lot of comments. I would never be able to manually moderate them all, not to mention the fact that comments moderatated tend to extinguish conversation between commenters. It’s not “naive” to allow comments on a blog, it’s intentional.

  35. John Rothko says:

    Since you have to check your comments anyway to filter out the bad ones, I do not see why moderating them upfront would be a problem. Especially blogs with open commenting are prone to be misused by spammers and now that they have find you out, you are going to have to monitor your comments very closely from now on. Therefore, I think IBL builder has point although he puts it rather bluntly.
    But I understand where you come from, moderating conversations doesn’t work encouraging, that is absolutely true.
    However, you can get around this problem by allowing trusted commenters to publish automatically. There is a setting for it this. In other words: if you have approved a comment for the first time, the commenter will then be able to comment next time without the need for approval. Secondly, you can filter comments that have more then one link. That’s another firewall for spam.
    Hope this helps?

  36. SEOPlay says:

    Excellent job paying close attention to your traffic. Even better job communicating this information in a clear, concise manner. Makes me worry about massive comments and the time necessary to manage them all.

  37. Jason says:

    Interesting post, I didn’t really think about the negative effects of Comments on your sites SEO, however I agree with most people here that say it was a text filter.

    Also search results can vary if a Data Center is having problems, sometimes searching the same keyword yeilds different results because it depends on which data center you’re hitting. I’d say it was the words (Which I’ll omit) ticking off the safe filter.
    This makes me want to write about it myself on http://green.cx but I think I’ll just link you instead! You did such a good and through writeup!

  38. I had a similar experience with a travel blog I ran – I found that if you don’t have the time to check things really really carefully its better to disable comments entirely – its so easy for this sort of thing to happen and for traffic to take a hit.

    At least things bounced back and you spotted the issue.

  39. Very well written. This is the kind of information that is useful to those want to increase their SERP’s. Keep up the good work.

  40. Fargham says:

    quite interesting, but congrat. you got traffic back

  41. Seo Trends says:

    Great review. thanks for sharing this very forwarded idea.i use twitter only for this. search engine marketing tips´s last blog.

  42. how your page in SERP within 24 hours!!!!!!

    great interesting read. Thankz

  43. tageek says:

    Did you have to submit a reinclusion request, or did rankings restore on its own?

  44. Tonnie is on the ball. I’ve seen this happen to one of my sites.

    Spammy comments are all over the shop on the web, on some well estalbished websites.

    I may have missed something here, but i find the assumption that you blame ‘spammy comments’ appearing as the problem, when what really sticks out is the sexual text that’s been dropped.

    Sometimes, things are simple. The algo is a whole is complex, but it’s made up of a lot of simple parts. I think this is one of them.

  45. Eblogger says:

    Did your page totally disappear from the SERPs, or it was pushed down for a number of spots? I have seen some -60, -70 penalties on some keywords on my site, although there was no comment spam involved. I still have to figure out why.

  46. Seo Pakistan says:

    How does google decide which backlinks it is going to show when you type in “link:www.domain.com”. Basically, I have many backlinks recorded with google but in different niches. Google doesn’t recognize any of my backlinks, even those with strong PR, in the same niche. Why?

  47. jack says:

    Very impressive and good work ! But when I look the last image, the visits have already started to grow up before you clear the comments ! Is it normal or what?

  48. Hi Jack,

    Yes I realise that screenshot represents all search traffic, not just Google – so when Google dropped the page, obviously the traffic dropped a lot but there were still a few entries coming from other engines.

    Also – the second day’s traffic would be Google recovering after the 7am fix.

    Hope that clarifies and thanks for taking the time to leave a comment!

    Richard

  49. trancoso says:

    I think that spammers think that a backlink is a backlink, especialy from high PR sites, no follow or do follow doesn’t matter for them, it’s all about the juice …

  50. Really interesting, clear article. And referenced again today by Rand at the SEOmoz site.

    Quick question, in your screenshots of the SERP’s, the top results from http://www.virtualbox.org/wiki disappear in the second example. Do you think there was some other factor influencing the ranking of your page (such as an algorithm update)? The timing of the resumption of traffic once you removed the offending comments obviously leads to the same conclusion, regardless. Thanks!

  51. kevin pike says:

    … or just manually approve first time comment users. It’s a small pain, but keeps the regulars happy without adding a captcha.

  52. Alex Cortez says:

    Painful reminder as to why some automatic processes just don’t do the same that human eyes can. Like Kevin said, maybe just do captcha for first-time users, surely a little hassle worth going through in order to participate.

  53. roshan says:

    can’t imagine a human would enjoy captchas, unless it is some funny captcha game or stuff like that. But a very insightful article nonetheless. Read about it from seomoz.

  54. I would also like to know why you linked to the reconsideration request tool.
    Are you trying to say that commenting on blogs may cause a site to be penalized?

    I hardly ever comment on blogs, unless (like now) I really feel compelled to.
    So if comment spam can indeed trigger a penalty, does that mean bluewidgets-com can comment spam blogs, while pretending to be greenwidgets-com and get them penalized?
    If so, can greenwidgets-com do anything to protect themselves from rogue competitors?

  55. Evan says:

    Saw the artible or on seowiz feed. It’s really difficult to engage users if they have to wait for their comment to be approved it feels regimist and non-genuine so in the way Captcha’s are better.

    Having said this Captcha’s take time and patience from the user. If I would have to choose I would use comment approving and explain clearly “due to risk of SPAM please enter the following….” and make sure the CAPTCHA is easy to read.. Who want’s to enter a captcha more than once?

    Thank you for the article!

  56. Liam Kiggen says:

    Read this article and then immediately went to my wordpress blog that I had not checked in a while and deleted a bunch of spam posts. Also downloaded latest version of Askimet.
    Thanks for the heads up

  57. ray says:

    Just read your post as well. Traffic to my site dropped like rock recently. Just so happens that a lot of comment spam was recently (past two weeks) posted. Just finished deleting most of it. Will be interesting to see if traffic picks back up quickly. Thanks for the post. =)

  58. Michael Wall says:

    Richard, interesting in the last couple of days I’ve come across 2 websites that have been injected with spam links to pills sites. One was on the homepage of a site, with 10’s and 10’s of links. The other was on a forum, that had obviously been hacked.

    Both sites & pages had been cached by Google weeks ago, though there was no obvious drop in rankings. The sites ranked where you’d expect them to rank.

  59. Hi There

    Examples always appreciated!

  60. Madhu says:

    Simply just wished to say I genuinely respect your work on this blogging site and the top quality articles you make. These type of post are usually precisely what keeps me personally going through the day. I found this post right after a good companion of mine suggested it to me. I perform a little blogging personally and I am always pleased to observe others adding quality information to the online community. I will absolutely be following and also have bookmarked your web blog to my facebook account for others to view.

    http://www.visitymyhub.com

  61. HP Bryce says:

    This is exactly why I do manual approval. I do not get nearly the amount of traffic yet as you do. Have you switched to Askimet yet? I know it is popular but I still have to do manual.

  62. I am surprised you were penalized so swiftly. Good thing you figured it out just as quick. Captcha is a pet peeve of mine (at least the squiggly ones that make me admit my eyes aren’t what they used to be). Unfortunately looks necessary these days.

  63. I am both amazed at the speed you got dropped as well as the speed you got re-included. I had no idea that PageRank is so dynamic? I am still recovering from loss of ranking due to missing ‘alt-tags on my photography blog, got lost in image searhc quickly and not yet back up..

    anyway, I agree that captchas are afwful and totally useless unless you enjoy employing people in Bangladesh & India (though I am sure other countires are catching up).

  64. I read irst time this type o information.. That is very great post..

  65. Hany B says:

    Hey Richard,

    At the time, did you try turning off adult filtering from google to see if it showed up in the search results?

    It may have not actually been dropped from the index but just marked as “adult content” and filtered by google’s safesearch filter. The default setting is moderate which means no explicit images only. But if there was quite a large amount of explicit text, alarm bells might start ringing.

    Good piece of information there!

    Cheers,

    Hany

  66. Rocky says:

    I found this from a link on SEOMoz. I’m not surprised by the speed. It could have been coincidence or Google might be going there everyday because of PR. This is troubling news for link buyers as they now have to find sites that don’t sell these spammy links.

  67. Mário Luan says:

    Nice point. But, why google doesn’t give link value for link building based on comments and give a big attention to spam-comments?…

  68. Tobias says:

    Totally agree with mike – fist-time approval is a great way to keep active commenters happy and the spam-bots away…

  69. tauranga says:

    in my opinion, commmenting is only good for a website if its a community website or a help blog or some other blog that requires feedback that others need to see. otherwise comments are always spammy

  70. John Vantine says:

    Woah… Never dawned on me that comment spam could effect your rankings – though it makes perfect sense. Thanks for sharing this… Def need to keep a closer eye on Akismet for some of my sites.

  71. Dimitris says:

    My website exprerienced a drop in unique visitors from 1200 per day to 200 per day in 10 days. My spam levers went up from 40 per day to 1500 per day… I deleted all the comments… and installed CAPTCHA. Lets see what happens..

  72. calvin says:

    I appreciate you taking the effort to prepare this article. Great line up. We will be linking to this on our site. Keep up the great writing.

  73. Thanks for this Great post. i think i have to check careful spam comment now so that Google will not be penalty to my site.

  74. Tom says:

    Thanks for the insite. Just about to implement WP on our website and this will make me a bit more cautious about security and moderating comments

  75. Richard, I have only discovered this post now through a web post but wanted to say it’s really great. This is a useful guide to the importance of managing the user-generated content that people post on your site.

    I always recommend to clients that they should not auto allow anything – always take the trouble to approve things that are acceptable and disallow anything that’s not. Considering at least 99% of comments in blogs are spam, it’s safe to say that you aren’t going to upset anyone if you reject 99% of the comments you receive. Your site will be better for it. I’d rather have no comments on a post than a ton of crap.

  76. ketan raval says:

    Google wants everyone to work manually .. apart from itself :) Nowonder .. if you receive significant view for this page after receiving google webmaster warning message :)

  77. Mally says:

    That’s some crazy stuff… I should make sure that my Akismet is turned on..

Comments are closed.