Some of you have pointed out that you’ve been seeing 500 errors when trying to post comments ot this site over the last few weeks. I’m painfully aware of the problems as I’ve been seeing the same thing when I’ve been posting entires.
But I think you’ll find that the problems have gone away. I’ve put some changes in place that seem to have fixed all of these issues. Please let me know if you are still seeing problems.
If you’re not interested in the details, then stop reading this entry now. If you wait a few minutes I’m about to post something that takes the piss out of the Daily Mail (again).
So here are the gory details. The problem was, of course, down to comment spam. I’m still getting hundreds of spamming attempts every day on this blog. None of them ever get published, so the spammers are wasting their time completely, but that doesn’t seem to bother them. Even though I’m not publishing the comments, the Movable Type comment handler still gets called for each of these requests and even just ignoring these comments takes enough processor power that the system is often brought to its knees and needs to be rebooted.
When Movable Type 3.34 was released, one of the major new features was support for Fast CGI. Fast CGI dramatically cuts down the amount of time it takes for a web server to respond to a request. So a properly configured Fast CGI setup would hopefully prevent my server being swamped by comment spam requests.
But it didn’t work out quite like that. Oh, it cut down the processor load alright. But something was broken somewhere in my Fast CGI configuration and that gave us those annoying 500 errors. I poked at it a bit every once in a while but couldn’t fix the problem.
Yesterday I got so annoyed about it that I removed the Fast CGI configuration. And within minutes my processor load was through the roof somewhere. But I remembered reading somewhere about an Apache module called mod_security which can be used to drop certain HTTP requests much earlier in the request cycle (so they use up less processor time).
As mod_security is pre-packaged for my system a quick “yum install mod_security” followed by “service httpd restart” was all that was needed to get the benefits. And the benefits have been huge. My processor load has remained low and I haven’t seen the MT comments program filling the process table. And that’s just with the default set of mod_security rules. There are endless possibilities for tweaking and improving them.
So if you’ve got an MT installation and you’re suffering from too much comment spam, I strongly recommend installing mod_security.