Referrer Spam - Day 3
In the continuing saga of Douglas vs. the blog referrer spam, I think I can finally say that while the war continues, the initial battle is over. And the war is more of an Internet war, being fought by system administrators around the globe. In any case, for now I have given up trying to stop the referrers at the firewall level, though I have some ideas that I may pursue in my spare time. Until those ideas get implimented, I’ve settled on an Apache-based solution, in addition to the Roller changes mentioned yesterday.
Thanks to some Google searches, I landed on Dave Child’s page on blocking referrer spam using the Apache’s mod_rewrite. You can see the details on Dave’s site and I’ve taken his list and updated and modified it. If anyone wants my list, I will be updating it regularly and would be more than happy to share.
To see the way it works, simply go to an ’Airplane Fetish’ page that I set up. Apache doesn’t look at the page contents but instead looks at the referring URL, that in this case will be sent along with the browser. Apache will catch the “-fetish” in the referring URL and instead forward you to a friendly page explaining what just happened and giving you a few options. I did this on the off chance that my rules catch something that isn’t really referrer spam, such as the link above which is clearly just a page written by a guy who loves airplanes.
If you have referring turned off in your web browser, you won’t see anything but then again I’m not trying to stop that. A typical referrer spam looks like this in the logs:
216.203.40.167 - - [18/Jan/2006:14:48:41 -0700] "GET /roller/page/downey?catname=/Games HTTP/1.1" 200 854 "http://www.some-bad-domain.com/keyword1/keyword2/etc/keyword-N.html
The bad guys are trying to get their domain, www.some-bad-domain.com, to show up in my referrer list with the hopes that either someone viewing the page will click on the link or that when Google will slurps up the contents of the page, it will see the link to some-bad-domain.com and increase that domain’s rank in their search results.
The irony of this whole thing? We don’t ever have referrer lists on our blogs. Nice.
Wed, 18 Jan 2006 14:55 Posted in Technology
