Introduction to OStatus: Atom-based federated messaging

I’ve written up the little introduction to StatusNet’s new inter-site federation protocol; for now you can also find this article on the StatusNet wiki. I’ll follow up in the next couple days with demos of the subscription UI and subscribing to Google Buzz feeds as native OStatus data sources. — brion

What is OStatus?

You’re about to learn what OStatus is and how it’s going to make communications between StatusNet sites work better!

Some of you may remember the bad old days when you couldn’t send e-mail between services like AOL and CompuServe. StatusNet/Laconica sites have used a custom protocol OMB to set up subscriptions between sites, but OMB requires special support to be used with non-StatusNet services and is hard to extend.

By building on top of shared technologies already in use and being rolled out, we can go beyond that… our new effort, with better compatibility between sites, is named OStatus.

The basics

If we step back a bit, we find that Atom and RSS feeds are already a lingua franca between sites. Virtually every service provides such feeds, giving a way for other tools and services to read someone’s outgoing posts.

File:Basic-feed.pngAt its simplest, a feed is a list of recent posts with some author attribution.

But the fun of social networks is in real-time interaction — if your updates don’t get sent to your friends right away, it’s just not the same.

Luckily, that exact problem has been solved recently, with PubSubHubbub (PuSH). In the PuSH system, a site can subscribe to updates for a feed from a hub server associated with that feed. Whenever a new post is created, the publishing site pings the hub, and the hub sends out just the new posts to all the subscribers.

OStatus is built on that simple base: each participating site produces Atom feeds of updates and uses PuSH subscriptions to send relevant updates to other sites:

File:Push-delivery.png

Alice posts a message on identi.ca; a fragment of her Atom feed is PuSHed out to the other StatusNet sites that have users following her posts. Each of those sites is responsible for routing the message to individual subscribers.

The real beauty of it is that at this point we already have something useful, without anything StatusNet-specific. In fact you can already subscribe to someone’s public Google Buzz feed as an OStatus remote user, and they haven’t done anything special for us!

The good stuff

PuSH is a low-level piece of the puzzle; it’s just a way for services to send updates to each other. For a fully interactive social network, you need that human touch… you want to know who’s subscribing to you, which of your posts they like, and you want to get their replies to you.

For this we’re making use of more extensions that build on top of Atom feeds: Activity Streams, Portable Contacts, and Salmon.

Activity Streams extends Atom feeds to describe social “activities” more specifically than just “ok here’s a post”. Entries are marked with a verb (post, follow, favorite, etc) and more detail on who or what’s being acted on. We’ve extended this further using elements from Portable Contacts to provide profile information:

Extended author information beyond what Atom’s native <author> element provides.

Salmon was created as a way for blog aggregators to send comments back to the original blog server, “like salmon swimming upstream”. We’re using this channel to send replies as well as other user-to-user events:

File:Salmon-delivery.pngBob replies to Alice from his home site; since she’s not subscribed to him, the reply is sent back to identi.ca as a Salmon ping.

When your friend on Example.Net subscribes to your Identi.ca account, Example.Net sends an Atom entry with an Activity Streams “follow” verb to your Salmon reply channel on Identi.ca. This lets Identi.ca know who subscribed to you, so their profile gets added to your subscribers list and you get a notification that they’re following you.

If your Example.Net buddy gets social network overload and decides to unsubscribe from you, another notification comes through about the unsubscribe, and we can drop them from the list.

Other events can be sent over this channel as well, such as when your friends on other services mark your posts as a favorite.

Groups

One of the most annoying limitations with OMB 0.1 was that there’s no support for joining a group on another service. You might have noticed that groups have Atom feeds of posts just like users do… if you were wondering if that means groups could be run over OStatus too, you’re correct!

As a user on Identi.ca you can join a group on Example.Net without setting up an Example.Net account. Group posts from Example.Net will appear right in your inbox on Identi.ca just like any local group, and you can even post back to it.