Recently in rss Category

It finally looks like The Sun have finally fixed their RSS feeds. It's only three months since I first noticed the problem. In that time I've emailed them a number of times on the subject. They haven't bothered to reply to my mail, so I don't know if they even read them, or if someone in their web department finally noticed the basic error that they'd made.

I say that they've fixed the error. All I know for sure is that they've corrected the URLs in the feed so you now get absolute URLs that work outside of the Sun's web site. As Martin has pointed out, they've still done this in a spectacularly half-hearted manner and they have probably lost most of the RSS traffic that they had built up over the previous years.

And whilst the most pressing problems in their feeds seem to be fixed, it's worth noting that the feeds they're publishing now are far from perfect. Running their latest news feed through a feed validator (something that any sensible feed publisher will do) shows that there are still quite a few interesting errors. At the current rate, I expect them all to be fixed sometime around the middle of 2012.

Two weeks ago I wrote about how the new, "improved" RSS feeds from The Sun are, in fact, completely broken. In that two weeks nothing has changed and the feeds are still broken.

Much as I enjoy seeing the Sun web team making a fool of themselves like this, I would actually like it even more if they would acknowledge their mistake and fix the problem. To that end, I've emailed them about it three times in the last two weeks. But all my mail seems to have been ignored. I've had no response and the problems haven't been fixed.

I'm pretty sure that the most critical problem (the broken URLs that are included in the feeds) is something that could be fixed in two minutes by someone who has access to the right template files on the Sun's web server.

So, given that I can't get any response from the contact address advertised on their web site, does anyone have any other suggestions? Do any of you know anyone who works in the Sun's web team or do you have any other avenues that I could try.

Or shall we just all sit back and laugh at the Sun for getting it so wrong?

It's been a while since the "sensational soaraway" Sun started publishing RSS feeds of their stories. I'm subscribed to a couple of their feeds (it keeps my blood pressure up) but I noticed a couple of hours ago that the feeds I was subscribed to no longer exist.

It seems that at some point in the last few weeks they have completely revamped all of their feeds. The details of the new feeds are on their site. Unfortunately the new feeds have been designed by someone who apparently knows very little about how RSS is supposed to work. These best example is that the links within the feeds are all relative instead of absolute - by which I mean that they don't include the server address. For example, one story in the current news feed contains the URL:

• /sol/homepage/sport/article420662.ece

where it should be the full URL

• http://www.thesun.co.uk/sol/homepage/sport/article420662.ece

Relative links only work for links within the same site. RSS feeds are (almost by definition) supposed to be displayed on other sites and therefore relative links won't work.

Having discovered this, I decided to check the feed with the online web feed validation tool (something that the developers really should have done for themselves) only to find that they really haven't done very well at all.

Earlier this week, Martin pointed out that the Daily Star have also started to publish RSS feeds, so I was planning to do some work on my newspaper feeds page this week. Looks like I'll have to do some work on the Sun section of that site as well.

Update: I was just looking at Martin's post about the top 100 UK newspaper web feeds and I noticed that the most popular Sun feed (their news feed) had 12,000 subscribers (and that's just in Google Reader). The figures are for the old feed. As the old Sun feed now just returns a 404 error, the Sun have just potentially lost 12,000 readers. RSS feed addresses are as important as any other URL on you web site. It should be as permanent as you can possibly make it. If you change feed URLs for some reason then you should put redirections in place so that your old readers can still find you.

This change gives every impression of being carried out by a complete amateur. I hope the Sun didn't pay too much for it.

Martin's article today about the Daily Express web site reminded me that it's been some months since I looked at my list of Newspaper RSS feeds. As the list is created by screen-scraping the individual papers' web sites, it's no surprise that it all goes out of date as the sites are redesigned and updated.

And sure enough, it was a real mess. When I ran the program that generates the pages, about half of them were broken. But it wasn't too serious, and after half an hour or so of tinkering with regular expressions, it all seems to be working again.

But all in all, it's a good lesson in why screen-scraping is a really bad idea. This would be far easier (in fact it would pretty much be unnecessary) if the papers took the next step and released OPML files of their feeds, rather than free-form web pages.

Anyway, it's all back again now. Please take a look and let me know if I'm missing anything obvious.

If you publish an RSS feed on your web site, then it's important that it continues to work. If you accidently break it, then you could be missing out on visitors.

I just happen to have an example handy.

London Eating is a useful site that carries reviews of London restaurants. For example, here's the page for my current favourite restaurant - Lamberts. If you view that page in a browser that understands such things (like Firefox or IE7) then you'll see the orange RSS icon indicating that the page has an associated RSS feed. That feed is here. Notice how both of these URLs contain the number 3295. I assume that's the unique number that identifies this restaurant in their database.

So far so good. But this is where it all starts going wrong. If you subscribe to that feed in your RSS reader of choice then you'll see a new entry appear each time a new review is published. This is what you'd expect. But the entries have three problems with them that range from annoying to making the feed all but useless.

Firstly, the entries don't show the full review. They just contain an extract. That's a bit annoying, but not too unusual. Many sites do that, so that you have to actually visit the site (and see all their advertising) in order to see the complete article. I'll just note that they had full feeds then I wouldn't have found the rest of the problems.

Secondly, each entry has exactly the same URL associated with it. So each review doesn't have it's own unique URL, they all share the URL of the page for that restaurant. It would have been easy to give each review on the page an HTML anchor and have the RSS URL going to that specific point in the page. Again, not a major problem - but it'll probably have some of the RDF fanboys gasping for air.

But the final problem completely overrides the previous two and makes the RSS feed pretty much useless. If you open one of the links in the Lamberts RSS feed you'll see this:

ADODB.Field error '800a0bcd'

Either BOF or EOF is True, or the current record has been deleted. Requested operation requires a current record.

/detail.asp, line 15

If I wasn't a bit of a techie, that error message would have sent me running from the site, never to return. As it is, I just grimaced and investigated further. And it didn't take long to track down the problem. The URL in the RSS feed is wrong. It contains the number 24075 instead of the number 3295 that you'd expect from our earlier experiences.

I presume that 24075 is some other identifier that is stored in the site's database. And that they are publishing the wrong identifier in the RSS feed. Obviously I haven't tried every URL in every RSS feed that they publish, but I've checked half a dozen or so of my local restaurants and they all exhibited the same behaviour.

I first noticed this a few months ago. And at the time I emailed the site owners and explained the problem. But I got no response and the problem hasn't been fixed. It's incredible that they haven't noticed the problem. It suggests that they added RSS at some point because they heard it was cool, but that they don't really understand what it's about.

Also, it would be nice if giving their site an invalid restaurant identifier didn't return an ASP error. It might be nicer if it said "sorry but I can't find that restaurant" or something like that.

As always, testing is key.

Update: About an hour after posting this entry (and less than an hour after sending a link to them) I got an email from the people at London Eating telling me that the main problem was fixed (and a quick test verifies this). They're even looking into fixing the two minor problems that I mentioned.

I love it when the web reacts that quickly. Kudos to the London Eating techies.

This article in Wired points out that the consumer release of Windows Vista will almost certainly see the use of RSS reach a tipping point.

Save the date: RSS will go mainstream on Jan. 30, 2007.

That's the day Microsoft's Windows Vista operating system hits retail shelves. Internet Explorer 7, the first version of Microsoft's web browser with native RSS support built in, ships on the Vista desktop. As users upgrade to Vista and IE7 and discover the convenience and power of RSS, we can expect a huge upsurge in the popularity of syndicated information on the web.

It's sometimes hard to remember just how far ahead of the curve you are if you use tools like RSS. But the coming weeks will see thousands of people seeing that little orange icon in their web browser for the first time and things can only get more interesting from here.

I've just noticed that The Sun has started providing RSS feeds. So now it's possible to get even more racist, sexist and homophobic bile delivered direct to your RSS reader.

Interesting to read how they try to explain the concept of RSS in words of one syllable for the benefit of their readers.

I'l get it added to the UK newspaper RSS list soon. I'll probably do another quick scan to see if any other papers have dragged themselves into the 21st century since the last time I looked.

I've made a start on building a directory of all of the RSS feeds available from UK national newspapers. It's online at dave.org.uk/newsfeeds. If I've missed anything then please let me know.

Also, if anyone can work out why my Javascript filtering code doesn't seem to work on IE, then I'd love to know what's going on.

Update: I've now added links to various OPML files. And (having cleaned it up a bit) I've released the source code.

Update: Blimey. We've been Winered. We are not worthy :)

More and more web sites are publishing RSS feeds of their useful data. But if you're going to publish RSS feeds then it's important to get it right. The Transport for London site is a good example of how not to do it.

Whilst trying to get the latest news on the current Northern Line unpleasantness, I ended up on this page on the TfL site. "Oh look," I thought, "an RSS feed. I'll add that to my Bloglines subscriptions." But it wasn't as easy as it should have been to track down the actual address of the feed.

My first guess was that the link labelled "RSS news feed" would link to the feed. But that turned out to be a link to a page that explained what RSS is. My next guess was that the orange "XML" icon was a link to the feed (that is, after all, the way they usually work). But no, that was just a graphic with no link at all. That, in itself, is a major break with standard practice. Then I looked for the "autodiscovery" headers that can be added to a web page to point to related RSS feeds. They were missing too.

At that point I was stumped. I'd exhausted all possibilities. Realising I had probably missed something, I retraced my steps. Remember the first link that I looked at? The one that contained an explaination of what RSS is? Well on further investigation it also contained links to not one but two actual RSS feeds. Finally I was able to add the feed I wanted to my feed reader.

Now I consider myself pretty clued-up on how this stuff works. But it took me a good couple of minutes to track the feeds down. The majority of users would probably not have my tenacity. TfL might be wondering why they aren't getting many subscriptions to their RSS feeds. I think it's obvious why.

I don't think it's hard to follow current best practices when implementing features like this. I wonder why so many companies get it so wrong. As long as sites have non-standard ways to access these features, users will just get confused and not use the features.

Interesting article from the Telegraph about the rise of RSS and its (potential) impact on news media.

To the consumer, the main benefit of RSS services is that they make receiving news more efficient. Instead of looking at one news website, then another, and then another, each time looking for stories that are interesting, RSS pulls news from the sites for you, and can do so thematically.

If you want business news from Sky, the BBC and the Telegraph websites, but without the fashion or gardening articles, you can have it. Simply choose the relevant feeds, and leave out the others. You can then go straight to stories without having to go to news organisations' home pages. RSS also allows users to pick up news from unconventional sources such as weblogs, or blogs, which are opinions, reports and diaries published by individual internet users.

(via Simon Waldman)