Michael W. asks:

I see all the work on comment spam, how about referrer spam? The wonderful Referrer Karma tool for WordPress practically eliminates the problem, but we on Textpattern have nothing like this. There are only so many variations one can manually add to an htaccess file (mine is 230 lines so far) before its a losing battle of bandwidth. Can the spam plugin process work for these as well? It looked very tied to comment spam.

There is a user-centered and a technical answer to that question:

The user-cenered answer is: log statistics are not a vital and central feature of Textpattern CMS, whereas comments often are; Comments are usually publicly viewable and thus comment spam requires often immediate effort on the part of the publisher to remove those comments. Spam referrers on the other hand are only shown on the private log page viewable by the Site Publisher, so the only action one must take is to ignore those referers.

Also be aware that while it was always possible and very easy to write a plugin against referrer spam, it was more difficult to write anti-spam plugins and their scope would have been limited. That’s why we worked hard for 4.0.3 and improved the infrastructure for writing anti-spam plugins that tie in well with the site and require little effort to develop and use.

The techincal answer(s): Leaving a comment is an interactive process which is controled-by and specific-to the application that is used, i.e. Textpattern. So when comment spamming became a reality, it was obvious to expect the application to be able to do something about it.

However referrers are part of the standardized HTTP-Protocol, we can’t just go in there and change that to force a reload, ask a question, enter a captcha or do other user- or application-centric things. With referrers it’s a simple take-it-or-leave-it situation, it’s residue of normal web-traffic. They are provided as is, and you can only choose to ignore/block/filter them. That limits the possibilities a lot.

The plugin you mention is in fact just that, a black- and white-list. So the effort shifts from maintaining webserver-configuration files to maintaining applications specifis files. I guess the harder part of it, is deciding on the interface (what to allow the user to configure and what not). Anyway, it would be easy to develop such a plugin for Textpattern (and it always was), but apparently there is just not as much interest in it.

Also, because referrers are part of the HTTP-Protocol referrer spam can — and on some hosts already is — being taken care of at other levels, for example using mod_security at the webserver-level, or at the analysis-level in log-statistic-software. And if you follow the discussions around those blacklisting/rule-approaches, you’ll see that while it’s possible to catch a lot of the referrer spam, it’s almost inevitable to also generate a considerable rate of false positives, which is when real harm might be done (in comparison to ‘unwanted’ referrers in private logs, you could be affecting users).

So, if there is sufficient interest in something like this, I am sure a plugin will eventually be written. Textpattern CMS certainly makes it easy to do so.