Comments are the lifeblood of the blogosphere

I spent the latter half of last week with my dad (photos). I did not bring a laptop, nor did I use any of his computers to access the internet. The trip was about dad, not about dad between e-mails.

When I returned to New York City, 193 comments awaited me in the moderation queue. 191 were spam. Some concerned a young lady. Others promoted medications. Two of the 193 comments were actually relevant to my site’s content, although they were trackbacks, not comments. (By the way, Wikipedia, which is it? TrackBack, with an intercap, or Trackback, without? Wikipedia’s trackback entry has it both ways.)

I use Askimet to control comment spam, and although it missed the 191 spam comments previously mentioned, it did flag as spam an additional ten comments, eight of which were spam. The other two were actual reader comments—the only real comments that came in while I was away. Askimet works for most users. Nothing works for me. But I digress.

Executive Summary: Of 203 comments received in a three-day period, two were comments (falsely flagged as spam), two others were trackbacks, and the rest were spam, although 191 of them were not identified as such. If comments are a site’s lifeblood, my site is having a stroke. (Which, by the way, was a popular verb in 42 of the spam comments I received.)

If I wrote more frequently, I would not get less spam, but I would enjoy a higher proportion of actual comments. I wrote every day, several times a day, for years here before comment systems, let alone blogging tools, were available. These days I have less time to write here or anywhere. But I will write more, promise.

I would get much less spam if my site were less frequently linked to and visited, but who wants a less-linked, less-visited site?

I would get no spam if I turned off comments, but I would also get no comments. And comments, real comments, are good.

Or so they tell me.

Comments off.


[tags]blogs, blogging, blogosphere, comments, spam, commentspam[/tags]

53 thoughts on “Comments are the lifeblood of the blogosphere

  1. I agree, Captcha should definitely help, but I’m surprised that Akismet and Bad Behavior aren’t working better for you.

    I occasionally get a false positive, but I’ve never had spam get through.

  2. I use Akismet myself and was inititally very, very pleased with it (I’ve been using it for nearly a year now). But in the last month or so, it’s been missing a LOT of spam. I don’t know if it’s in dire need of an update or what, but it’s defintiely not working like it used to. Glad to hear it’s not just me. :)

    I can’t bring myself to turn off comments on my blog. They’re just too valuable to me. The discussion and the readers are why I blog. If I was going to turn comments off, I’d just keep a personal diary. Blogging is about the community and discussion, as far as I’m concerned. Yes, spam is annoying, but I’m prepared to deal with it — the comments are important enough to me.

  3. As a fellow WordPress user I would be interested to here wether or not you have changed the look and or feel of your administration area, you know, to make comment administration easier on the eye?

  4. As a fellow WordPress user I would be interested to here wether or not you have changed the look and or feel of your administration area, you know, to make comment administration easier on the eye?

    Nope. I use the admin as-is. Works best that way.

  5. That sucks and I do feel your pain. Comment spam is crap, it is the scourge of the blogoshpere that is for sure.

    Myself, I have had pretty good success with Askimet – although it started bugging me to have to monitor that all the time too. So I set out for methods to cut down on the amount of stuff that even Askimet was getting. I now employ a few different techniques that are behind the scenes a bit and that have effectively elliminated by spam problems.

    For instance, I use hidden fields that if populated with something different than expected drop the comment to null/void instantly. If you are human and are fillling out hidden fields, let me know – as I guess I don’t want to miss your comments (rolleyes).

    I also use a bot trap that if entered by a roaming bot that doesn’t adhere to nofollow and robots.txt stuff, then it gets the “door” and that door is then locked behind it.

    Since I’ve done these things, my daily spam as disappeared and now I only get a post every now and then and then that is caught by Askimet – so I feel much better now and my life has returned to normal. I know that it is only a matter of time until the spammers catch on but for now I am happy.

  6. I’m with you… real comments are nice. It’s like getting a handwritten letter in the mail. For what it’s worth, I feel silly leaving comments that just say “I agree!” and usually I can’t think of anything else to contribute to your posts. You’re just too cool for me, Mister Zeldman.

    Oh, and I’m with Jeff Croft, too: Akismet was working wonders for me up until a month or so ago.

  7. Six Apart always spells it TrackBack but I do not because I find that unattractive. I tend to call it trackback. On the other hand I respect their spelling of Movable Type and cringe when I see Moveable Type although they were intelligent enough to acquire both domains back in the day.

  8. I experienced an upsurge in comment spam on my site about two months ago. Initially I wrote it off as the normal ebb of the www, but the surge continued and grew. I was also using Akismet and found myself wading through hundreds of quarantined comments every morning. Two weeks ago I discovered the Bad Behavior plugin. Since installing Bad Behavior—I still have Akismet installed—the daily deluge has dropped to about 30. Bad Behavior claims to have stopped 886 attempts. I’d say it’s worth checking out if you’re not already using it.

  9. Never really felt much like leaving comments on blogs. I just like to lurk and read :) Love your’s though JZ.

  10. Comments fuel the communication and bloodflow of the blogosphere, and yet it’s a diseased fluid. I get about 20 a day on my blog, and it enrages me. Simply because I feel it is an attack on me personally. Sure, it’s just another web site, an easy target, but some of use feel our blogs are extensions of us, whether business or personal. It’s the equivalent of salespeople walking into your office during a meeting and NOT LEAVING. That’s happened to me.

    Plus, it’s now impossible to have a rational comment conversation about Viagra. Not that I’d want to, but still.

  11. Do the spammers “hope” you approve one of the comments on accident? I dont get it. I’m assuming most comments are moderated now. How the hell would any of this spam actually get posted? Oh wait.. or are they targeting the person doing the moderation knowing that they will get read? blah… who knows.. stupid spammers…

  12. Akismet’s been working great for me, but the volume of incoming spam that it collects is utterly mind-blowing. I think right now it’s at something like 180 comments marked as spam. I have the same problem as you, in that I don’t write incredibly often (although I do plan on writing often while I’m in Europe this and next week), don’t get many (real) comments, but still like comments and want to keep them in.

    That said, I would be VERY interested in learning Todd Lambert’s scheme. I never check beyond the first page of Akismet. Who wants to devote that much time during the only life they get to reading comment spam? So theoretically, I could actually be missing lots of real comments. (Although somehow I doubt it.) So a system that further siphons through the comment spam that I do receive would be welcome.

  13. As Todd Lambert said above, and I read in an article in the March issue of Sys Admin magazine, hidden form fields are (currently) an excellent means of getting rid of most spam. The trick is to put a text input field on the form, and then use CSS to hide it. Human readers will not see it, but spam bots will. Thus you can safely discard any message that fills in the field. This has several advantages over CAPTCHA – there is no obstacle to the user filling in the form, it is much more friendly to screen readers and other accessibility tools, and there is much less load on the server.

  14. You (and me) are long in the tooth enough to remember how the blogging world was like without commenting features. And hardly anyone complained. That is, before the blogoshpere pundits claiming that “a blog without comments is not a blog” came in. A line of thought that, needless to say, I don’t really feel like sharing.

    Comments, however, enabled blogs to become conversational entities, and that’s a great thing in its own right. Oftentimes the comments can be more interesting than the post itself. However, it just takes a small group of selfish, rotten scumbags that care about nothing but money at any price, to turn a valuable resource into a major nuisance and piss everyone else off in the process.

    Which just goes to show the truth of a personal statement – that of the man being his own worst enemy.

  15. But I will write more, promise.

    I think I’ll hold you to that. I love it when you write on your blog. It’s less fun when you write in your books instead. The shopkeepers keep trying to charge me for them…

  16. CAPTCHAs are really annoying.

    I have a home made CMS, and just ask commenters a simple question: “Are you alive?” Bad responses just throw an error which clue in actual human beings to an appropriate answer.

    Maybe if my site had more traffic, someone would invest the time to hack this simpler turing test. But then I’d just change the question. Absent a scary amount of artificial intelligence, it would take 30 seconds to defeat hours of programming time.

    We have complicated, invisible back-end systems to filter spam from email because of the technological structure and long-standing cultural traditions surrounding the medium. The act of sending email is not, fundamentally, interactive. Mail gets queued up and could be delivered hours, or days later, or never at all. People expect to be able to send a letter and walk away. To have to legitimate their identity– again, hours or days later– seems onerous.

    None of these factors are true for a collection of form fields on a website. Response is immediate. People are used to checking for and fixing errors if they occur.

    Askimet seems like a good idea if you want moderation— to separate good (real) people from bad (real) people before the bad people get to speak. But for spam it seems less reliable and more time consuming than just putting a little intelligence in your form.

    Meh, maybe I’m just too lazy to implement another API. I still haven’t gotten around to trackbacks.

  17. I want to commend you on going away without your laptop and accessing other computers. I think we all need to ‘step away from the mouse’ more often but unfortunatley in the web world it is difficult. I am trying to do that more, but it isn’t easy.

    Cheers! melissa :)

  18. I’ll take a look at your key to see if anything unusual is going on, you may have just been a victim of the brief DNS outage a few days ago. The newer versions of the Akismet plugin have a button to recheck all comments in the queue, which can quickly clean things out after a problem on your side or ours, and we hope to make this re-checking automatic at some point in the future.

  19. Turning off your comments isn’t the answer. Comments are what make blogs come to life. They allow authors to stimulate conversation and involve their audience. Put up with Viagra and his friends and keep the comments coming!

  20. I use Expression Engine which uses a combination of a centralized blacklist (from pmachine) and a captcha doo-dad. I’m not a huge fan of the captcha thing, since I haven’t modified it for accessibility yet, but those two solutions have worked *very* well. Since I deployed those, I get *zero* comment spam. I peek at the apache logs and I can see the idiot spam-bots trying to POST, but it doesn’t work. *whew*

  21. I’ve varied my approach on handling comment spam a few times now, going between home-grown solutions like referral checking and strict email validation (bouncing a message off the mail server to make sure the email address actually exists — to working with the Akismet library. I haven’t noticed much of a change in Akismet’s reliability for me personally, but I also haven’t been using Akismet for that long.

    Either way, even though it’s annoying and time consuming I still think it’s worth it. As others have pointed out, there’s so much that readers can offer through their comments and removing that feedback and discussion would be a huge loss.

  22. The problem with CAPTCHAs as they’re typically implemented is that they’re not accessible. Blind people can’t read them at all. Visually impaired people have significant difficulty. Older people have difficulty.

    I’ve found the hidden form fields to work well on my e-mail contact forms. Hide them with display: none and you avoid potential accessibility problems because most screen readers these days don’t read display-noned text. I give the hidden form a nonsense name.

    One solution I implemented that I haven’t seen mentioned anywhere else is that I change the field names on my comment form every day. I wrote a little perl program that grabs some words at random from /usr/share/dict/words, then assigns those words to the comment form field names and writes them to a PHP file that my comment form then reads in to assign the field names. Any automated spammers that visit my site and grab field names have a window of 24 hours at most before their comments are rejected, which has proven enough to stop pretty much all the automated comments I receive. Human visitors using the actual form on my site don’t run into problems (unless they visit just before the names are changed and submit after the names are changed; I could fix this but it’s an edge case so I haven’t bothered). I’m not aware of any plugins for WordPress that do this, though.

  23. Like Todd, I use hidden fields. I also use a link throttle as it seems that 99.9% of spam will contain a bunch of links. The blog software I use puts these comments in a moderation queue. Since 2003 I have yet to have a “real” comment in the 100 or so comments that appear in the moderation queue every day, so now the comments in the moderation queue are deleted every night.

    This does not prevent spam from a person who manually types a spam comment into my blog and stays below the link threshold. I’ve had two of those in the last three and a half years.

    Finally, none of these measures prevent “hate” comments which I delete as is my right. Luckily, this has only happened to me three times.

  24. I use Movable Type on one of my websites, so I don’t know if this would work for WordPress. What I do is when a spam comment shows up, I change the path to the cgi that runs the MT. I’ve been using this trick for over a year and have had to do it only twice. The site does not have a huge amount of traffic, so that may be a factor. But, previously, I was getting comment spam every few minutes.

  25. Hi Jeff,

    The silly question on your comments section should work like a charm… bots are stoopid anyway.

    One thing that I have been doing lately, is actually turn comments off after X amount of time… say 21 days… Are the spamments submitted from very old comments? I am very sure this is the case… I have found that spambots often miss recent posts… Older posts get bombarded with spam, probably due to posts propagation over search engines… The older the post the higher the chance for it to get spammed.

    Posts in blogs like yours, usually get comments on the very first few days after it has been posted, then, they become ‘Reference Material’.

    Should someone have anything else to add… heck… they will for sure send you an email or message through a contact form.

    But I may be on crack… my blog doesn’t even get 1/1000th of the hits that you get in one day… so maybe I am being naive.

    What do you think?

  26. Regarding TrackBack in camel case versus trackback as a word, I’d have to say the latter should be fine from the perspective of how a screen reader might “say” the word — it should speak it properly due to the word’s structure, but writing it in camel case would guarantee this.

    Regarding the number of comments requiring your attention, I’d really have to suggest you look at adding the Bad Behavior plugin as it’ll stop many of the comments from reaching the Akismet queue. For example, on my blog in the past week I’ve had roughly 864 comments wind up in the Akismet queue, forcing me to look at them (albeit very quickly). Bad Behavior caught an additional 1828 and did so passively, not forcing me to waste my time. Moreover, I do find both plugins play nicely together.

  27. Hi Jeffrey
    I tried Askimet for a week or so, then went back to SpamKarma which does the trick for me. Misses a manageable few comments, only two false positives.

  28. Nothing tastes worse than a big breakfast of spam after a long weekend away. I’d also like to see blog/CMS developers provide better built-in protections against spambots (alas, human spammers are a whole other can of ham).

    As developer of a website CMS, I’ve managed to keep spam off my plate by piling up several anti-bot techniques, some of them already mentioned above, in combination with Akismet. Individually, they’re only half-measures, but together they’ve been effective. For example: field names change every second, hidden fields trap bots, JavaScript is required to submit to the correct address, and a custom challenge question provides a simple turing test.

    I wrote up the details in Seven Habits of Highly Effective Spambot Hunters (warning: labored hunting metaphor ahead!). I also recommend Ned Batchelder’s Stopping spambots with hashes and honeypots.

    Spambots are only half the battle, though. None of these measures will stop living, breathing viagra vendors and porn peddlers. Analytical tools like Bad Behavior and Akismet have to act as backstops. From the sound of it, though, even these fine filters are feeling the strain of the spam arms race.

    Unfortunately, Jeffrey, your site (like other high-traffic blogs) is likely to be a testing ground for new comment spam techniques. Services like Akismet may miss the new pattern a few times before adapting to identify the new strain. In a sense, you’re taking a spammy bullet for the rest of us…

  29. I’m surprised no one’s plugged Spam Karma yet. Let me fill that gap! It rolls a whole bunch of plug-ins into one extremely clever spam-fighting machine. It’s such a complete solution that I barely even notice that I get comment spam any more. Oh, but I do:

    Total Spam Caught: 22057 (average karma: -584.93)
    Total Comments Approved: 284 (average karma: 56.02)
    Total Comments Moderated: 110

  30. Spam Karma II for WordPress works great for us. We get hundreds of spam comments a week, and I stopped looking at them, since there were never any false postitives. Apparently, in borderline cases, it triggers a captcha, but I can’t get it to trigger it myself, so I’m not sure what sort of captcha it is.

    Manual commenters who are pushing their site can’t be stopped with automated software, but they are rare since it’s so labor-intensive.

  31. Melissa said:

    I want to commend you on going away without your laptop and accessing other computers.

    My dad’s been coping with health problems. I needed to show up for him 100%. If I brought a laptop or used his computers to connect, I knew I’d get sucked into my world of clients, conferences, and communities. I did the only logical thing.

    Mike said:

    I’d really have to suggest you look at adding the Bad Behavior plugin as it’ll stop many of the comments from reaching the Akismet queue.

    I use Bad Behavior in conjunction with Askimet. Without them, I’m sure the situation would be much worse.

  32. I use Bad Behavior in conjunction with Askimet. Without them, I’m sure the situation would be much worse.

    No doubt :-/

    You might want to try a plugin by Mike Jolley aptly named the Comment Spam Stopper. I haven’t tried it myself, but it’s based on some of the tech I use in my contact form plugin so I know it’s a solid solution.

    I also like the “honeypot” method mentioned by Josh Clark, above, where an input is hidden (by positioning, not input type), and if populated returns an error. You just need to be sure to provide a descriptive label so users who do “see” it understand it’s not to be populated. That’s one I use on my contact form as well (turn off styles to see the label and input).

  33. Don’t know what cms this blog is built around, but after years of using Askimet alone I found that while it was still capturing thousands of spam comments, it was letting more and more through. My solution was to add a couple of layers of protection… I check for actual keypresses in the input field, I run the results through a plugiin called spam firewall, and then finally they hit askimet… The combination has been so effect I’ve even turned off moderation… which seemed almost scary at first, but ultimately has been liberating… As an added bonus because some of these steps happen before the comment actually hits mysql my server has been much happier.

  34. Is there anything web-related to do in NYC on the 26th or 27th of this month? I’m want to take the train up for the Neil Gaiman events at the Moth but I need to justify the time off to work. So far my attempts to find a lecture, class, conference, or discussion group has only turned up a GUI developers meetup.

  35. Strange. Akismet has been a life saver for myself as well as all the folks I set up with WP blogs. It has rarely been wrong in passing judgment upon comments.

  36. I turned off comments on my long ago.

    Receiving no comments (with “comments on”) was like having a cell phone with no one to call.

    I got rid of my phone in January.

  37. This comment is generated automaticly via random brain-fingers-keyboard generator and has no blue pills inside.. :-)

    Have you tried comment preview and submit AFTER preview and not without it? (of course previewed text must be the same as the submitted one).. It worked very very nice for all my friends around Czech IT blogosphere..

  38. The only thing worse than comment spam is writting about it and getting more. It seems like no matter what tools you use, the weasels still get through. More visitors means more links and more links means more spam. There’s no way to win unless you set up an ultra secret blog and only post once: “I get no comment spam because I posted only once and I have no comments”. Of course your site will hit Digg and then you’ll be screwed. There’s no way to win.

  39. Dear Mr. Zeldman,

    I come from an Eastern European country which has experienced war at the end of the 20th century. I know what it feels like to run into the basement because of aerial bombardments and what it means to wait for bread under sniper fire. Maybe that’s the reason why I find your posts about hearing police sirens in NYC frightening while your dog is sleeping annoying.

    Yet, I bought the first edition of your Web standards book (at a very high price, where I live in) and implemented it in my work. It has paid off, and sometimes, it was a stumbling stone because of managers and marketing (I can bet that the same line of thinking you’re being confronted with even today). I love standards. I love the Web.

    Yet, as much as you brag about women being NOT included into Web design business, you NEVER EVER got out of the US frame. There was not a single post about problems web developers and designers are facing in the world outside the US. Everything is so much about content, solidarity and genuine content, yet all of that never gets out of US state borders.

    It is a closed world. And I will be completely honest with you: I resent that. I follow your “web-standards evangelism”, yet I am excluded from it, like millions out there are, who don’t happen to own an US passport.

    I don’t own a credit card, yet I WILL buy the expensive english-bookstore second edition of your book, although I downloaded a pirated version of it. Because I believe in everything that you have put in the making of this book.

    Your bragging about how it is the right thing to publish original content, not being the next Web 2.0 thing is comparable to me bragging about how there are not ANY persons whatsoever on your blog, flickr list etc. that are not outside the US. Sorry Jeff, I respect you VERY much, but I did NOT see any event apart in Europe recently. Picture this: “to gain a deeper understanding of web standards and emerging best practices. Be inspired by fresh ideas and new directions. Join the greatest minds and hottest talents in web design today.”

    Well, according to you, an Event Apart REALLY IS an Event Apart. It boils down to Web standards people who happen to have US citizenship. And the show goes around the US. Well if the Web is supposed to be accessible to everyone (as you tell in your book AND as I tell people around me, believing in a TRULY universal Web), it should include everyone. But it does NOT. Like EVERY Web 2.0 service against which you wrote, against every proprietary browser-rendering technology you wrote, the Web should be accessioble to everyone. Yet every single thing that you write about includes the topics that revolve EXCLUSIVELY around US firms, US Web authors, US conferences and so on.

    My point is: Why bother to talk about universality of the Web and promote Web standards if it is just a US thing? You can brag all you want about examples from here and there but the fact is that you are completely unaware of what is happening outside this “so heroic” US web-standards arena.

    I am sad. I am disappointed. Yet, I will buy your and a load of other books to learn from. And I won’t just download and print the pirated versions, although the prices for them here are inhuman, although I am excluded from the possiblity of paying with a credit card, although the discussion about these crucial Web subjects is going on, I won’t participate, because no one cares. All the references are geared toward the exclusively US “event apart” events etc.

    A 32 KB DSL connection is a problem??!? How about being on a standard modem connection and pushing Web standards in spite of being rejected by a lot of firms who pay well, here, in the non-US?

    Dear mr. Zeldman, I respect you, and I thank you for sharing the knowledge, but my disappointment has surpassed my enthusiasm. Pushing out Web standards in a poor, counter-personnel mined country while paying outrageous sums to learn about standards, in spite of being able to download pirated versions (at slower and FAR more costly speeds than your oh-so-poor-me post brags about), but sticking to morale sucks royally.

    I will buy your 2nd edition book. But. I will never EVER read your blog again. Or look up if there are some new books by you. I am so bitter. Please don’t write about your fears in NYC while hearing sirens anymore. There are a lot of us who take for granted much more worse than that, and we don’t even brag about DSL connections and such. I would give my right arm to have a connection like yours, so much clients unaware about Web standards etc etc.

    Be decent, at least. PLEASE. There in fact IS a world outside your US sphere. Yes, really, there is.

    And no, I do NOT hate you, your country or anything for that matter. I just wish to point out that you are no better than people about whom you think that they care about money only and not web standards, quality content etc. You don’t care about any of that outside your narrow life circle anyway.

    Sirens, NYC, oh, cruel world.

    Flattened houses, minefields, web standards, oh cruel world.

    Farewell, Mr. Zeldman.

  40. I’m glad someone else mentioned Spam Karma. I’ve been using it for years long before Akismet was created and I don’t have too many proplems. I don’t get that many real comments, but I don’t spend a lot of time moderating spam. Spam Karma just knows so much of it is pure crap and I don’t even see it.

    I don’t double check near as much as I do with email looking for false positives. I think I finally registered for a account so I could use Akismet, but I seem to stick with Spam Karma.

    Plugin comments are popular, I had so many (OpenID, Gravatars, Subscribe To Comments) that I had some cross mojonation so I got rid of a bunch of them. Has Gravatars 2.0 been launched yet? I looked in to some alternates to it, such as MyBlogLog but I don’t think these little pictures are worth the headache… OpenID would be nice but it never seemed to work correctly…

  41. I am not sure if Tony’s comment was addressed, but the problem with using CSS to hide it is that screen readers may pick up the hidden element, and visually impaired users would fill it out. Thus making all comments by users who are visually impaired as junk.

    My comment I before I read some of the comments was that on my blog that I started in October, I have received nearly 1,500 spam comments. I think it is absurd.

    As an aside, Jeffrey, you may want to look at the tabindex ordering of first part of the comment form, it jumps a bit…

  42. we have tried everything, to stop spam but it wont, and i think i found a reason as we throw too much hate to spam , why not we all start to love spam and respect them and care for them and expect the fact that they are nice little programs who are just doing there job and then maybe we can stop them ?

  43. Why are you asking Wikipedia how TrackBack is capitalized? Decide and correct the entry, or bring it up on the Talk page.

Comments are closed.