Tag Archive for '.htaccess'

How to block comment spam

Published
by
Tom Raftery
on 24/7/2006
in .htaccess, Comment Spam and Technology
. 15 Comments

Like all bloggers, I find comment spam to be a constant annoyance. There are many ways to mitigate the problems it causes however and using the following techniques means that this site is subject to almost no comment spam.

Use WordPress’ built in comment spam tools -

  • In WordPress Options -> Discussion, fill in the list of common spam words - words in this list automatically cause a comment to go into the moderation queue. I use the following list.
  • Also use the Comment Blacklist field. Populate this very carefully. Any comment containing words in this list are nuked automatically. No notification. No way to get them back. Gone. This is the list of words I have in my blacklist.
  • I have checked the “Comment author must have a previously approved comment” field as well. This is a very simple but very effective tool - regular commenter’s are able to leave comments and see them appear instantly; new commenter’s comments are held for approval and if they are not spam, their comment appears in short order and subsequent comments appear immediately.
  • And I use WordPress’ built in anti-spam plugin - Akismet.

I also have a custom .htaccess file which stops a lot of spamers cold before they reach the site at all. Excercise extreme caution with .htaccess files as they can take your entire site down. If you are not sure what you are doing, I have written a few explanatory articles on .htaccess files previously. If you are still not sure what you are doing, put the .htaccess file down and walk away very slowly!!!

Finally, I use plugins called Referrer Karma and Bad Behaviour which help significantly by stopping bots from accessing your site to leave comment spam.

Having implemented these techniques ensures that my site stays free of comment spam without having to moderate all comments and without having to implement CAPTCHAs. CAPTCHAs are those horrible badly drawn images of combinations of letters and numbers which some people put on their sites to stop spam. CAPTCHA’s are evil*. Stop using them. Now.

* The American Foundation for the blind has written many times about how difficult Captchas make browsing for blind or partially sighted people and the W3C in a report on Captcha’s said:

A common method of limiting access to services made available over the Web is visual verification of a bitmapped image. This presents a major problem to users who are blind, have low vision, or have a learning disability such as dyslexia.

If you enjoyed this post, make sure you subscribe to my RSS feed!

Blocking trackback spam using .htaccess

Published
by
Tom Raftery
on 7/4/2005
in .htaccess, Blogging and Trackback Spam
. 14 Comments

This morning I received a trackback spam. It pointed at a rubbish domain - ohuudfghj.com, and came from ip address 172.164.210.50 using User Agent Mozilla/4.0 (compatible; MSIE 5.5; Windows 98; Win 9x 4.90).

I took a look at Spamhuntress’ site and sure enough she has a post warning that a trackback spam run is about to get underway imminently.

Then I checked out my raw log files and found several entries from this User Agent, all from different IP addresses so banning the IP address would be useless to block the spam.

Consequently, I added the following line: 

SetEnvIfNoCase User-Agent ^Mozilla\/4.0 \(compatible; MSIE 5.5; Windows 98; Win 9x 4.90\) spammer=yes

to my .htaccess file.

Although this may seem a tad drastic, I trawled through my raw log files and couldn’t find any legitimate entry for that User Agent in my logs.

Be aware that if you intend to use this code, you need to use it in the context of the surrounding code in my .htaccess file (i.e. follow the code with 

deny from env=spammer

if you are uncertain, be sure to check out my .htaccess file).

You can test the efficacy of this code by going to the Wannabrowser site, entering the User Agent into the HTTP User Agent field, your site’s address in the Location field and clicking the Load URl button. You should get a 403 result if the code is successfully blocking this User Agent.

UPDATE: Diane let me know that this code was too strict as it was blocking her and she isn’t on a Windows 98 PC. Spamhuntress pointed to a script to block access to Trackbacks - basically you use this script. I have been using the script and haven’t received any trackback spam since I installed it.

If you enjoyed this post, make sure you subscribe to my RSS feed!

Using WordPress as a CMS - More problems sorted

Published
by
Tom Raftery
on 21/3/2005
in .htaccess, Blogging and WordPress
. 6 Comments

WordPress is the Content Manangement System (CMS) used to produce this site but shoehorning it to produce a complete site can cause unforseen problems. For example, my Permalink structure was denying me access to my awstats folder!

Why? - this is because I’m publishing the site using WordPress 1.5 and it’s mod_rewrite (created by the permalink’s structure) incorrectly assumes that the Awstats folder is part of the WordPress site (because the Rewritebase is set to /) and when it can’t find Awstats within WordPress, I get dropped into a 404 page.

I tried accessing the folder using the ip address instead of the domain but couldn’t get in - my ISP subsequently told me this wouldn’t work - that I need to come in under my domain for awstats to function.

To sort this out, I inserted the following code above the code WordPress inserted in my .htaccess:

RewriteCond %{REQUEST_URI} ^/awstats/
RewriteRule .* - [L]

Hat tip to Niall for helping out with this.

If you enjoyed this post, make sure you subscribe to my RSS feed!

Comment spam plugins no longer required!

Published
by
Tom Raftery
on 8/3/2005
in .htaccess, Blogging, Comment Spam and WordPress
. 45 Comments

I have written many posts on my battles with WordPress comment spam but all that appears to be coming to a very satisfactory solution. I am now no longer using any comment spam plugins and I have stopped moderating comments on this blog.

How did I get to this enviable position? Well, it has been a long road and I have learned loads about WordPress along the way.

I started down this road by trying various comment spam plugins with different degrees of success. However, none were really satisfactory. The best one was WP-Hashcash - best in that it was most transparent to the user - but it requires commenters to have Javascript turned on in their browser. So I kept looking for another strategy to eradicate this scourge from my blog.

I upgraded from WordPress 1.2 to WordPress 1.5 (the current version) - WordPress 1.5 has a number of anti spam comment features natively built in.

Of these, I have set the number of links allowed in comments to 3 - any more than that, and the comment is auto-moderated.

I have populated the blacklist with a short list of words (just over 40) - any comments containing these words are automatically deleted - boom! No notification to me, no notification to the commenter.

I have written a custom .htaccess file which blocks a lot of potential spam commenters at the gates. Instructions on how and why I set it up are here.

And finally, I have installed Dr. Dave’s plugin Referrer Karma. I know, I know, I said I didn’t have any comment plugins, but I don’t. Referrer Karma is a referrer spam plugin which just happens to work like my .htaccess file (but much more elegantly) to block the bad guys at the gates.

The combination of these measures has allowed me to turn off moderation on the comments on my blog - and so far (one week later) no comment spam has made it through my defences. I’m not saying the war is over but, so far, I seem to have won this round.

If you enjoyed this post, make sure you subscribe to my RSS feed!

Block hotlinkers but allow some sites remote access to images using .htaccess

Published
by
Tom Raftery
on 5/3/2005
in .htaccess, Blogging and Linux
. 4 Comments

In a previous post I explained how to create a .htaccess file to stop remote image linking (hotlinking) and bandwidth theft - however, there are some situations where you might want your image files linked to from remote sites - how do you make exceptions for these sites?

The code to block all sites from hotlinking to your images is, as follows (see my previous post for a detailed explanation of the code):

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?tomrafteryit.net [NC]
RewriteRule \.(png|gif|jpe?g)$ - [NC,F]

To allow Google, AltaVista, Gigablast, Comet Systems, and SearchHippo translators and caches to be able to link to images we need to use the following code:

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?tomrafteryit\.net [NC]
RewriteCond %{HTTP_REFERER} !^http://216\.239\.(3[2-9]|[45][0-9]|6[0-3]).*(www\.)?tomrafteryit\.net [NC]
RewriteCond %{HTTP_REFERER} !^http://babel.altavista.com/.*(www\.)?tomrafteryit\.net [NC]
RewriteCond %{HTTP_REFERER} !^http://216\.243\.113\.1/cgi/
RewriteCond %{HTTP_REFERER} !^http://search.*\.cometsystems\.com/search.*(www\.)?tomrafteryit\.net [NC]
RewriteCond %{HTTP_REFERER} !^http://.*searchhippo\.com.*(www\.)?tomrafteryit\.net [NC]
RewriteRule \.(png|gif|jpe?g)$ - [NC,F]

And obviously, everywhere you see my domain (tomrafteryit.net) in the code, substitute in your own domain.

If you enjoyed this post, make sure you subscribe to my RSS feed!

Using .htaccess to minimise comment and referrer spam

Published
by
Tom Raftery
on 12/2/2005
in .htaccess, Blogging, Comment Spam and Referrer Spam
. 23 Comments

I have been using my .htaccess file to stop comment and referrer spam on this site and it has been surprisingly successful (so far!). How do I create a .htaccess file capable of greatly reducing comment and referrer spam?

Firstly, I use Awstats to analyse visits to my site daily and I use Spam Karma to help control comment spam. Both applications give me information on spammers visiting my site.

Awstats gives me a list of the referer sites - this list contains those sites which are trying to spam my referrer logs. I monitor those sites and as new ones appear I add them to my .htaccess list in the form:
RewriteCond %{HTTP_REFERER} \.domain\.tld [NC]
where .domain is the domain trying to spam my site (psxtreme, freakycheats, terashells, and so on) and the .tld is the top level domain the site is registered to (.com, .net, .org, .info, etc.).

So, for instance, in the case of the spammer coming from the smsportali.net domain, I have added the following line to my .htaccess code:
RewriteCond %{HTTP_REFERER} \.smsportali\.net [NC]
This will stop accesses from all subdomains of smsportali.net (spamterm.smsportali.net) to the site and the NC ensures that this rule is case insensitive.

In the case of comment spam, I have configured Spam Karma to email me every time it deletes a spam comment - this is becoming rarer and rarer as the .htaccess file becomes more and more effective. I have configured Spam Karma to include the server variables and request headers of a comment that is not approved in the email - this is one of the configuration options of this plugin.

Scanning these emails, I can see the User Agents being employed by these spammers - armed with this information, I added the following lines to my .htaccess file:
RewriteCond %{HTTP_USER_AGENT} Indy.Library [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Crazy\ Browser [NC]
RewriteRule .* - [F]
and this has greatly reduced the amount of comment spam coming through.

Also, Cindy alerted me to the fact that adding:
RewriteCond %{HTTP:VIA} ^.+pinappleproxy [NC]
RewriteRule .* - [F]
Will also catch a lot of the spammers.

I have a copy of my .htaccess file available for review (it is in .txt format).

NOTE:
For each set of rules in your .htaccess file, you need to finish with a RewriteRule - RewriteRule .* - [F] will give a 403 (page forbidden) to the spammers. Your last set of rules should end with RewriteRule .* - [F,L] - the L telling the RewriteEngine that this is the last line and to stop processing the rules here.

IMPORTANT WARNING:
the .htaccess file is a very unforgiving file. It has the power to make your entire site unavailable to anyone. It is strongly advised to read up on Regular Expressions and Mod_Rewrite (the Apache module which processes these commands in a .htaccess file) before creating a .htaccess file or modifying an existing one.

If you enjoyed this post, make sure you subscribe to my RSS feed!

Tom Raftery’s Social Media is Digg proof thanks to caching by WP Super Cache!