Dougal Campbell's geek ramblings

WordPress, web development, and world domination.

Testing a new spam blocker

I’ve been trying to snatch a few minutes here and there to work on the automated spam blocking system that I proposed last week. Earlier today, I finished up one of the components, and I’m currently testing it. It seems to be performing as designed, so far.

There are three pieces to the system, currently:

  1. ipupdate: A perl script which accepts an IP number as an argument. When run, it checks whether the IP already exists in its configured database table. If not, it adds it. If so, it updates the last access time and a counter associated with the IP.
  2. ipmanage: Another perl script. This one is run every five minutes as a cron job (with root permissions). This script grabs the current set of rules from ipfw,
    compares them against entries in the database, expires old rules, and adds new rules for IP numbers which have crossed the defined threshold.
  3. WPIPFW: A simple WordPress plugin. It merely checks a few spam indicators (connections from open proxies, blacklisted referer strings, comments posted and flagged as ‘spam’). If any of the indicators are true, it passes the client IP number to the ipupdate script.

I still need to tweak the auto-expire logic a bit. Currently, it only looks at the database entries to determine dormancy. But an IP blocked by the firewall won’t be able to trigger the conditions that update the database. I need to capture the IP accounting stats from ipfw and update the lastaccess field in the database if the IP is still seeing activity. Otherwise, all IP numbers will automatically expire from the database, even if they continue their (failed) attempts to contact my server. Though, they’ll automatically get blocked again once they re-cross the threshold of spam attempts.

It would also be moderately easy to get most email systems to call out to the ipupdate script. In fact, any service or program which allows you to call out to external scripts could use it. One good project (which I might also implement here) could be to scan the logs from your mail server, and automagically add/delete database entries based on SMTP rejections.

Unfortunately, this will only be useful to folks who have the ability to modify the firewall rules on thier server. Also, it’s currently hardcoded to work with ipfw, which is what my FreeBSD server uses. But it should be simple to modify it to work with other firewall systems and rulesets.

When I finish the current features and get it cleaned up a bit, I’ll release the code. It’s not very pretty, and I won’t be able to offer any technical support for it. But if anyone else finds it useful, that will be great.

I know that this isn’t a new idea, but I haven’t seen another implementation that broke things down into a simple generic system that other services can hook into easily. If anybody out there has seen something like this, post a link!

So far, it appears to be working for me. In the couple of hours since I activated the weblog plugin, it’s added over 60 IP numbers to the database. Of those, 18 have already been banned at the firewall level, and 8 of those have attempted approximately 110 more spam attempts (combined) since being blocked. Another 10 hosts are just one more spam away from being blocked. The next thing to be tested is the auto-expiration of spam hosts that have gone dormant. More as it develops.

About Dougal Campbell

Dougal is a web developer, and a "Developer Emeritus" for the WordPress platform. When he's not coding PHP, Perl, CSS, JavaScript, or whatnot, he spends time with his wife, three children, a dog, and a cat in their Atlanta area home.
This entry was posted in Blogs, Plugins, Servers, Spam, WordPress and tagged , , , , , , , . Bookmark the permalink.

6 Responses to Testing a new spam blocker

  1. Dougal says:

    Update: There are currently over 150 IP numbers in the database. 84 rules in the firewall. 77 of those are repeat offenders. Approxminately 10,000 spam attempts blocked. Woot.

  2. w00kie says:

    what kind of language are you using for these scripts of yours?

  3. Dougal says:

    The main scripts (ipupdate and ipmanage) are coded in Perl. It’s really a very simple bit of code. The main tricky bit is parsing out the current firewall rules. And that’s really not hard at all.

    The main thing that’s going to keep other people from using my code is that it requires your web server to have firewall support, and it requires you to have root access (in order to manipulate the firewall rules).

    On the ‘pro’ side, an admin with the resources could modify it to distribute things a bit. You could have instances of the ipupdate piece running on different hosts, triggered by multiple services (web, mail, security logs, etc). They could all be configured to send updates to a common database. And they could also be modified to retrieve firewall state information from another, possibly centralized, source (like a border router).

  4. Thomas Cloer says:

    Why reinvent the wheel? Spam Karma 2 in combination with Referrer Karma, both from drDave kills them all. I’ve been running them on my site for quite a while and nothing even comes close. You might want to check them out…

    But maybe you already know Dave’s plug-ins and think that you’ve come up with a simpler or better concept. Then i would of course love to test yours.

    Best regards, Thomas

  5. Dougal says:

    This is completely different from what SpamKarma does. I’m using an external tool to block repeated spam attempts at the TCP/IP level, before the web server ever sees the packets. This reduces load on the server, because Apache/PHP/MySQL never even know that any kind of request was even attempted.

    However, my method still relies on the existing systems to identify the first few spam attempts. Once a particular host has attempted too many spammy things, it gets blocked by the firewall, and nothing else on the machine is going to see any kind of traffic from that source until the block expires.

    So, SpamKarma (or any other spam detection tools) could be a part of an overall strategy along with my tools.

  6. Pingback: geek ramblings » SpamValve update

Leave a Reply

%d bloggers like this: