PubSubHubbub: Cool, but Hard to Say

By Jake • August 20th, 2009 • 14 Comments

PubSubHubbubPubSubHubbub, a 20% time project of two Google engineers, Brad Fitzpatrick and Brett Slatkin, launched a few weeks ago.

In addition to being a tongue-twister, PubSubHubbub is:

A simple, open, server-to-server web-hook-based pubsub (publish/subscribe) protocol as an extension to Atom (and RSS).

Parties (servers) speaking the PubSubHubbub protocol can get near-instant notifications (via webhook callbacks) when a topic (feed URL) they’re interested in is updated.

Essentially, your feed reader won’t need to poll all your feeds for updates periodically anymore, saving loads of resources. Each feed that declares a hub hands off updating subscribers to the hub, which pushes notifications when they’re available. Anyone can declare a hub for their feed to use by adding some code to the feed, and anyone can operate a hub.

The protocol will definitely speed up feed reading, and it has applications for any syndicated content and any frequency, e.g. it could be used for a micro-blogging or IM, in a much more efficient push updates mode instead of a persistent connection mode.

Check out the video from the Real-Time CrunchUp for a an introduction and demo.

There are several products using PubSubHubbub already, including FriendFeed, Google Reader Shared Items and Feedburner. Today, Google Alerts joined this list.

This blog uses Feedburner’s PingShot, so we are have been using the bootstrap hub hosted at Google App Engine for several weeks. If you check the source for our feed, you’ll find:

<atom10:link xmlns:atom10="http://www.w3.org/2005/Atom" rel="hub"
href="http://pubsubhubbub.appspot.com/" />

Did you notice updates were faster? I don’t really post enough for it make much of a difference. I suppose I should have noticed, since I publish the posts and can see how quickly they appear in Google Reader.

Rich will probably implement a hub for Connect to use inside the firewall too.

A few months ago, Connect was plagued by performance issues; it was falling over every day at 0200 Pacific time. Not ideal, since we’re all asleep at that time and couldn’t bring it back up for a couple hours.

Rich and Anthony pinpointed the problem. A ton of feed readers were polling for updates at 0200, causing Connect to die. They got this issue fixed, and Connect has been humming along fine for a while (knock on wood). Intrigued, Rich decided to analyze Connect logs to see how often feed readers were polling.

We were all astonished to find 1.4 million requests for feeds this month alone.

1.4 million is a lot for us, especially considering the relatively modest web metrics for Connect for this month, only about 65,000 pageviews and 5,000 unique visits so far.

We’re still trying to figure out what this means. It doesn’t feel like feed reading has suddenly become mainstream, but maybe . . . I’m at a loss.

Anyway, PubSubHubbub seems like the perfect solution to save our servers from millions of GET requests.

Heard about PubSubHubbub? Any other ideas on how it could be used? Are you updating your blog to use a hub?

Find the comments.

Update: This post took about five hours to get into my Reader. So, I guess Reader hasn’t finished implementing the protocol yet, aside from Shared Items.


Possibly Related Posts

  • http://nbrightside.com/blog Andy C

    I think the name is far too easy to remember and say and wish Google had used 'PubforLunchSubwayHubbleTelescopeBubbleboil'n'trouble' instead.

    Seriously, I think it's one of the more exciting and innovative changes in quite a while and PubSubHubbub will probably go a long way to delivering the backbone of this 'real-time Web'.

    I don't necessarily need my blog post to show up in Google Reader is less than 3 seconds flat and I don't use XMPP as my hub for all things but, one day, I might.

    Oh and, of course, it's the backbone for Google Wave which is the really interesting part and why Google will push it hard.

  • http://only4left.jpiwowar.com jpiwowar

    Heh. Looney Toons flashback. “Ehhhhhh….what's all the PubSubhubbub, bub?”

    Wonder if it makes sense for the OTN forums to use a hub. Lots of frequently updated content there, and feeds for every forum. Don't know how much of their user base follows the forums via RSS, though. Maybe it's just me. :-)

  • http://theappslab.com Jake

    Agreed, the potential applications are exciting. It's probably good to get it tested with non-critical things like feed reading first before branching out to more real-time applications.

  • http://theappslab.com Jake

    I think it makes sense for any application that facilitates communication. Real-time is the expectation we all have for communication right? I'm thinking of phone calls where you say something and have even a scosh of lag time. Or TV interviews over satellite where there's a pause between question/answer.

  • http://twitter.com/kottarainen Joonas Linkola

    “Taking away those two has increased load times quite a bit, so that’s good” -> I believe you mean *decreased* load times, right?

    PS. I NOTICED! (The downtime, that is.

    This comment was originally posted on Oracle AppsLab

  • http://theappslab.com/ Jake

    Decreased natch, thanks for proofing Sorry about the downtime. We keeping maxing out our memory on the host for some reason. No idea why.

    This comment was originally posted on Oracle AppsLab

  • http://twitter.com/oraclenerd oraclenerd

    well deserved break. enjoy.

    This comment was originally posted on Oracle AppsLab

  • Pingback: Taking a Break | Oracle

  • anonymous

    this protocol sucks the biggest dick ever! I fucking hate it and hope it dies a horrible fucking death!!!!! why make something to hard to fucking develop with and not god damn documentation!

  • anonymous

    this protocol sucks the biggest dick ever! I fucking hate it and hope it dies a horrible fucking death!!!!! why make something to hard to fucking develop with and not god damn documentation!