Local Planets

Over the last few years I’ve written a few times about how I’ve been building planets. A planet is a web site which aggregates web feeds on a particular topic and republishes them as a combined web site (almost certainly with a combined web feed as well). One of my earliest planets was Planet Balham which combines feeds about Balham, the area of London where I live.

The idea of using the internet to bring together local communities has been gaining a lot of traction recently, so I’ve been doing a bit of work on Planet Balham firstly to improve the design and secondly to make the content as interesting as possible. I’ve also promoting it a bit and, as a result of that work, someone suggested to me earlier today that a Planet Streatham might also be useful. For those of you who don’t have an encyclopaedic knowledge of London geography I should probably point out that Streatham is the area to the east of Balham.

I have a pretty good system in place for building planets quickly so in my lunch break I threw together a quick prototype for Planet Streatham. And then (because I was on a roll) I did Planet Tooting and Planet Clapham too.

Of course, the problem with building planets is finding good content. For Balham, I have some local knowledge and I’m pretty happy with what I have[1]. For Streatham, Tooting and Clapham I have less local knowledge and, to be honest, less inclination to research the subject. But I’ve discovered that you can actually build a decent local planet with just a few standard feeds. And those are what I’ve used for the new planets. I think these would be a good start for any local planet. The great thing about them is that it’s easy to customise them for any other location.

In all cases, I hope it’s obvious how to customise the  link.

Automated searches aren’t without their problem, of course. Since I’ve been following Google’s news search for “Balham”, I’ve learned more than I really wanted to about Nebraskan basketball player Chris Balham. But that’s only to be expected and the “real” results far outweigh the problems. Initial results indicate that the problem might be worse for Planet Tooting. Tuning the search terms – perhaps to include “London” – might be an improvement.

So I have basic planets for Streatham, Tooting and Clapham. I don’t intend to spread my empire any further. But I’d really like to see more local planets like this springing up. I’ve already had a couple of people contact me on Twitter about creating others. There’s already a Planet SE16, but there’s no reason why every part of London shouldn’t have one.

The technology isn’t hard. My planets are built with my own software, but I expect pretty much any language will have some kind of planet application available. I’ll write in more detail later this week about how I’ve built mine, but if I’ve inspired you to build one, please let me know and I’ll start some kind of directory.

[1] Still interested in adding more though, let me know if you know of a good local feed that I’m missing.

More Planets

Over the weekend I found time to rebuild the rest of my missing planets. I’ve resurrected Planet Balham (Atom), Planet Westminster (Atom) and Planet Doctor Who (Atom). They all have Atom feeds available as well.

This has been an interesting test of Perlanet (my simple planet-building program). When building planet davorg, I was only using feeds that I had some kind of control over. It was therefore pretty simple to ensure that the web page created was valid HTML (though, due to some bugs in the Perl modules I’m using, the same can’t be said of the Atom feed). But with these new planets, I’m aggregating feeds from all sorts of places and am seeing problems that I hadn’t seen before. In particular I’ve changed Perlanet to deal with the cases where the feed can’t be downloaded for some reason (I think that some of the MPs on my list have stopped blogging) and where the feed isn’t valid.

There are also plenty of examples of feeds that have some pretty mad HTML in them which are breaking the layout of the output pages. On Planet Balham there seems to be some broken HTML that is badly effecting the <div>s on the page, moving the Google Adsense block halfway down the page. Also, the second half of the page is currently in italics due, I suspect, to an unclosed <i> tag. On Planet Westminster there’s also some kind of problem which means that the names of the feeds change size halfway down the page.

So it’s clear that I need to add something to clean up the feeds. I’ll probably look at using HTML::Tidy or HTML::Scrubber (perhaps both). Expect some better looking pages in the next few days.