Regular readers will know that this blog has experienced some problems over the last few months. Since I last moved to new hardware this blog has been running really slowly and more often than not any attempt to publish content (whether a blog entry or a comment) has ended with a server error. This has made me less likely to write here and has made you far less likely to comment. Which is sad.
When the problem first arose I prodded at it a bit, but nothing I did made any difference. So I decided to ignore the problem in the hope that a Movable Type upgrade would fix it at some point in the future.
Over the last couple of weeks, that strategy has been causing me large amounts of grief. The problem is that any of the programs that publish content to this site are taking such a long time to complete that the connection times out. So what should take a few seconds is actually taking two or three minutes. Over the last few weeks the amount of comment spam I get has risen to new levels. And even though none of that spam ever gets published, each request runs the MT comment program. For about two or three minutes.
As a result, when I come under a comment spam attack (which is happening several times a day) I get dozens of copies of the comment program running simultaneously (because they aren’t finishing quickly enough). The load
on the server goes right up and all of the sites on this server stop responding. I’ve been having to reboot the server on an almost daily basis. Sorry if you’ve experienced problems with any of my sites being unresponsive.
On a separate front, I decided to move hardware (again). This was prompted by the fact that the people who I rent my
colo box from have started to support Centos as well as Fedora (they have Ubuntu too, but I’m far happier in rpm-world). Centos is a far better distribution for servers than Fedora so I’m in the process of moving.
Last night I started to move my blog on the hope that the new hardware and new operating system might somehow fix the performance problems. As part of the process I needed to dump the database that runs the blog on the old server. Some way into the dump process, MySQL told me that it couldn’t dump the mt_log table as it was corrupt. I logged in to the database server and ran “fix table mt_log”. The dump then ran successfully.
Then it struck me. Any time you publish content on an MT site, you write to the mt_log table. And the symptom I was seeing was that a process would complete all of its work
and then hang. And presumably all of the logging goes on at the end of the process.
So, hopefully fixing the mt_log table has fixed the performance problems. I was able to successfully rebuild the whole site in thirty minutes last night. Previously it would have taken hours and had a good chance of failing somewhere along the line.
We’re running on the new hardware and operating system too, Which, all in all, should mean that the performance problems are no more. I’ll start blogging a bit more and hopefully you’ll start commenting a bit more too.
And here’s a good tip for fixing MT performance problems.
Use the MySQL “check table” and “fix table” commands regularly. I’ll certainly be looking to schedule an automatic job to do that.
Sorry for any inconvenience. Regular service has (hopefully) been restored.