Server moved

Last night, my host finally relocated my server (as previously mentioned). The move occurred around 3am EDT. For some reason, apache didn’t automatically restart when the machine came back up, and I had to start it manually this morning. So the web site was down for about 6 hours. Also, DNS has changed, so if you were still unable to reach this site after about 9am EDT today, it was probably a DNS propogation issue.

Testing a new spam blocker

I’ve been trying to snatch a few minutes here and there to work on the automated spam blocking system that I proposed last week. Earlier today, I finished up one of the components, and I’m currently testing it. It seems to be performing as designed, so far.

There are three pieces to the system, currently:

  1. ipupdate: A perl script which accepts an IP number as an argument. When run, it checks whether the IP already exists in its configured database table. If not, it adds it. If so, it updates the last access time and a counter associated with the IP.
  2. ipmanage: Another perl script. This one is run every five minutes as a cron job (with root permissions). This script grabs the current set of rules from ipfw,
    compares them against entries in the database, expires old rules, and adds new rules for IP numbers which have crossed the defined threshold.
  3. WPIPFW: A simple WordPress plugin. It merely checks a few spam indicators (connections from open proxies, blacklisted referer strings, comments posted and flagged as ‘spam’). If any of the indicators are true, it passes the client IP number to the ipupdate script.

I still need to tweak the auto-expire logic a bit. Currently, it only looks at the database entries to determine dormancy. But an IP blocked by the firewall won’t be able to trigger the conditions that update the database. I need to capture the IP accounting stats from ipfw and update the lastaccess field in the database if the IP is still seeing activity. Otherwise, all IP numbers will automatically expire from the database, even if they continue their (failed) attempts to contact my server. Though, they’ll automatically get blocked again once they re-cross the threshold of spam attempts.

It would also be moderately easy to get most email systems to call out to the ipupdate script. In fact, any service or program which allows you to call out to external scripts could use it. One good project (which I might also implement here) could be to scan the logs from your mail server, and automagically add/delete database entries based on SMTP rejections.

Unfortunately, this will only be useful to folks who have the ability to modify the firewall rules on thier server. Also, it’s currently hardcoded to work with ipfw, which is what my FreeBSD server uses. But it should be simple to modify it to work with other firewall systems and rulesets.

When I finish the current features and get it cleaned up a bit, I’ll release the code. It’s not very pretty, and I won’t be able to offer any technical support for it. But if anyone else finds it useful, that will be great.

I know that this isn’t a new idea, but I haven’t seen another implementation that broke things down into a simple generic system that other services can hook into easily. If anybody out there has seen something like this, post a link!

So far, it appears to be working for me. In the couple of hours since I activated the weblog plugin, it’s added over 60 IP numbers to the database. Of those, 18 have already been banned at the firewall level, and 8 of those have attempted approximately 110 more spam attempts (combined) since being blocked. Another 10 hosts are just one more spam away from being blocked. The next thing to be tested is the auto-expiration of spam hosts that have gone dormant. More as it develops.

URL ABCs

Feeding the meme…

These are my URL ABCs:

Spammers should all DIE DIE DIE

I am so sick of the damn spammers. Spammers are teh sux0r. Spammers are a festering boil on the ass of the Internets. I wouldn’t let a spammer kiss my butt with a pair of wax lips from ten feet away. If I ever see a spammer bleeding in a ditch, I will not be a Good Samaritan, I will kick him in the head, cover him up with dirt, and leave him there to rot.

Over the past few weeks, the comment spam has been coming in so thick that I’ve had to start blocking IPs at the firewall level. I’m currently blocking over 40 IPs, plus an entire Class C block out of Mexico. My normal blacklist blocks prevent the spam from showing up on the blog anyhow, but they were coming in so fast and furious that Apache and MySQL were grinding to a halt under the load.

On top of that, the email spam has been spiking up, too. Which makes SpamAssassin and Procmail eat the CPU. I’ve added a couple of additional RBL checks to my anti-spam measures, and that’s helping, but not as much as I’d like. I’m thinking about implementing some much more aggressive measures.

What I want to do is to start tracking spam source IPs in realtime. When I determine that a blog comment or email message is spam, I’d add the source IP to a database. I’d update a spam count and modification date every time I receive more spam from that source. After reaching a certain threshold, I would automagically ban that IP in my firewall rules. The IP would stay blocked until a certain amount of time passed with no traffic at all, at which point it would be removed from the firewall.

This would ensure that any particular spam source would only get a very limited number of tries to waste my resources. It will also be a pain in the ass to implement, but at this point, I’m about ready to spend every spare moment that I can find to do it. I’ll keep everyone updated on my progress. If I can get it working well, I’ll release the code for anyone else who might be able to use it.

Server move

I’ve been informed that my host server will be relocated later tonight. So if you find that you can’t access the site for a while, that’s probably why. This will not be a migration to new hardware as mentioned previously, just a physical move of the existing hardware to a new location.

Update: It looks like the move didn’t happen last night after all. So just be warned that the site could go down at some unknown time.

There’s no place like home

Faced with foot-high grass in our back yard and a no-show lawn service, we bought a lawn-mower this weekend. Yesterday, I spent about three hours (including a couple of water breaks and a fuel refill) beating the grass into submission. When all was said and done, our lawn was looking pretty nice.

I’m really glad that we spent the extra money for a self-propelled mower. We’ve got about an acre of land, and that gentle slope in the back yard doesn’t seem quite so gentle when you’re pushing a hunk of metal across it over and over again. The extra-large back wheels and the 6.5HP engine didn’t hurt, either. If I had cheaped out and gotten a 4.5HP model, it probably never would have made it through some of the thicker clumps of grass.

I’m starting to feel like a real homeowner now 🙂

Mobile Wireless

Have you ever wished that you could have your own wireless network available, wherever you go? How about a mobile wireless access point, GPS tracking, and a webcam in your car? Would that do the trick?

The EVDO StompBox Project is just such a beast. So, if you’ve got about $700 and some time to blow, you’re all set!

[via: BoingBoing]

Server problems

I’m sure some of you have noticed that I’m experiencing occassional server problems. The host I’m on has been overtaxed, mostly due to spammer activity (both web and email varieties). There are times when the server load spikes up into the 20s, which is ridiculously high. To combat the problem, I set up a cron job which monitors the load level, and restarts apache and mysql when it goes above a certain threshold. However, for some reason, the mysql database server sometimes fails to restart properly. I’ve just added another monitor program which should fix that, so the database should never stay down for more than a couple of minutes. I still need to put something in place to do something more proactive about the email spam, though. Part of the problem there is that the volume of spam email is causing SpamAssassin and procmail to hog the CPU. I may need to enable some of the more aggressive SpamAssassin features.

I’ve been told that we’ll be getting some new hardware soon, so hopefully things will settle down more when that happens. It should also give me an opportunity to do some better performing tuning for the server.

Cake PHP Framework

If you’re like me, you’ve been hearing “Ruby this” and “Rails that” everywhere you go. But I don’t have time right now to be playing with a new development framework, however nifty it may be. Busy, busy, busy, that’s me. I’ve got my perl programming to do at the day job, and in those tiny little niches of time that I can nab for other things, it’s generally PHP for me.

But maybe now we can have our Cake and eat it, too. Cake is a PHP framework based on Rails. Features:

  • compatibile with PHP4 and PHP5
  • supplies integrated CRUD for database and simplified querying so you shouldn’t need to write SQL for basic operations (although some familiarity with SQL is strongly recommended)
  • request dispatcher with good looking, custom URLs
  • fast, flexible templating (PHP syntax with helper methods)
  • works from a website subdirectory, with very little Apache configuration involved (requires .htaccess files and mod_rewrite to work; these are available on most web servers)

So, if you’ve been wondering what the Ruby on Rails hoopla is about, maybe running through the Cake tutorial will give you a taste. One of the core ideas is to simplify the developer’s interaction with the database. Writing code to interact with the backend database is one of the most tedious things that a web developer has to deal with. It seems like every time we need to write even the simplest of applications, we end up re-writing the same functions over and over. Web application frameworks like RoR and Cake greatly simplify this process. You just set your tables up with a few standard fields, create a matching class which inherits from the framework’s database model, create a simple little view skeleton, and — BAM! — you’ve got your basic database interaction all wrapped up. The framework handles all the CRUD operations for you.

[via: fiftyfoureleven]

Happy Birthday, Ping-O-Matic!

Ping-O-Matic is one year old today. To think that my lowly little service pinger, originally created to help me gain an edge on BlogShares, is all grown up and sending out approximately 1 million pings per day! We’ve sent out pings on behalf of over a half-million individual blogs, and over 85 thousand in just the last week.

I haven’t been able to give PoM as much attention lately as I’d like. I still have lots of ideas that I want to implement, but it’s just hard to find time to do it, between the dayjob, the new house, spending time with the wife and kids, and preparing for the upcoming baby. But I keep hoping that someday soon I’ll be able to spend some time adding some of the new features I have in mind.