Federation protocol overview

The purpose of this document is to describe the communications that go on between Diaspora servers. Implementers of this protocol should be advised, though, that Diaspora is in Alpha, and as such, this document is not authoritative, and may lag behind the reference implementation. You can find an implementation of this protocol that you can reuse in this gem

Asymmetric Sharing

One important thing to note is that Diaspora’s notion of “sharing” is asymmetric.

In a symmetric “friending” relationship, neither party sees any posts from the other until one party makes a friend request and the other confirms.

In an asymmetric “sharing” relationship, if you start sharing with someone else, you have decided to send them posts. They may or may not choose to share with you as well. If they choose not to, you will see only their posts that they have explicitly marked as public.

Public posts are sent in salmon, just like all posts.

Core Diaspora Protocols

Diaspora servers communicate with one another in a variety of situations:

• When discovering information about users on another server.
• When sending information to people that you’re sharing with. That information includes:
  • Notification that you’ve begun sharing with them.
  • Posts that you’ve made.
  • Comments that have been made (by you or others) on one of your posts.
  • “Like”s that have been made (by you or others) on one of your posts.
  • Conversations (each thread in the inbox has an object representing it)
  • Messages (each individual message in a Conversation)
  • Profile information
  • Retractions of posts
  • Retractions of likes/comments

This document does not cover the semantics of each of the messages listed above. For a discussion of these semantics, see Federation Message Semantics.

Discovery

Diaspora pods MUST be able to discover users on other pods, given the other user’s webfinger address. For convenience, Diaspora pods’ user-interfaces MAY choose to allow users to search for users by name, searching through the list of names already known to the pod (such as local users). However, pods’ user interfaces MAY NOT allow users to find a person by name if that person has not marked themselves as “searchable”, in their hcard (see below).

If alice@alice.diaspora.example.com wants to discover bob@bob.diaspora.example.com, then alice’s pod must perform a Webfinger lookup of bob’s address. Webfinger is an open protocol. See the Webfinger protocol specification for the full details. However, we will summarize here. Note that bob’s webfinger profile does not need to be hosted by bob’s diaspora pod. Any webfinger server will do, so long as bob’s profile contains the elements necessary for Diaspora. See below. However, Diaspora pods SHOULD host webfinger profiles for their users.

Alice’s pod will first get the host-meta file from bob’s webfinger address. The host-meta file is located at https://bob.diaspora.example.com/.well-known/host-meta. In the host-meta file, alice’s pod will find a Link element with rel="lrdd", such as:

<Link rel="lrdd" template="https://bob.diaspora.example.com/?q={uri}"  type="application/xrd+xml" />

Alice’s pod will now transform bob’s webfinger address and replace {uri} with that. First, bob’s webfinger address is url-encoded. Alice will make a GET request to: https://bob.diaspora.example.com/?q=bob%40bob.diaspora.example.com

Bob’s webfinger server (which may be the same as bob’s pod) will respond with bob’s webfinger profile. The webfinger protocol might look something like this:

<?xml version="1.0" encoding="UTF-8"?>
<XRD xmlns="http://docs.oasis-open.org/ns/xri/xrd-1.0">
  <Subject>acct:bob@bob.diaspora.example.com</Subject>
  <Alias>"http://bob.diaspora.example.com/"</Alias>
  <Link rel="http://microformats.org/profile/hcard" type="text/html" href="http://bob.diaspora.example.com/hcard/users/((guid))"/>
  <Link rel="http://joindiaspora.com/seed_location" type="text/html" href="http://bob.diaspora.example.com/"/>
  <Link rel="http://joindiaspora.com/guid" type="text/html" href="((guid))"/>
  <Link rel="http://schemas.google.com/g/2010#updates-from" type="application/atom+xml" href="http://bob.diaspora.example.com/public/bob.atom"/>
  <Link rel="diaspora-public-key" type="RSA" href="((base64-encoded representation of the rsa public key))"/>
</XRD>

Let’s look at these elements in more depth.

Subject

<Subject>acct:bob@bob.diaspora.example.com</Subject>

The Subject element should contain the webfinger address that alice asked for. If it does not, then this webfinger profile MUST be ignored by alice’s pod.

Alias

<Alias>"http://bob.diaspora.example.com/"</Alias>

(this seems to be the pod location, but with quotation marks around it. Why?)

hcard

<Link rel="http://microformats.org/profile/hcard" type="text/html" href="http://bob.diaspora.example.com/hcard/users/((guid))"/>

Bob’s webfinger profile MUST contain a link to an hcard. The hcard contains personal information such as bob’s full name, a link to bob’s photo, etc. Refer to the hcard specification for a full discussion on hcard syntax.

Like the webfinger profile, the hcard need not be hosted by the Diaspora pod. In fact, it may be in a location that is distinct from both the Diaspora pod and the webfinger host. However, if the hcard is to be hosted by the Diaspora pod, then the url SHOULD be: http://bob.diaspora.example.com/hcard/users/((bobs-guid))

When a user creates an account on a pod, the pod MUST assign them a guid – a random hexadecimal string of at least 8 hexadecimal digits. The Diaspora pod SHOULD use that guid to create an hcard url, and SHOULD host users’ information as an hcard at this location.

Diaspora adds a field, entity_searchable, which contains an element with class “searchable”, which is either “true” or “false”. If the value is false, then pods MAY NOT allow users to search for bob by name, or through any method other than directly entering bob’s webfinger address.

Diaspora also includes a url field with the id of “pod_location”, which links to bob’s diaspora pod.

Here is an example of an hcard.

<div id="content">
<h1>Bob Exampleman</h1>
<div id="content_inner">
<div class="entity_profile vcard author" id="i">
<h2>User profile</h2>
<dl class="entity_nickname">
<dt>Nickname</dt>
<dd>
<a class="nickname url uid" href="http://bob.diaspora.example.com/" rel="me">Bob Exampleman</a>
</dd>
</dl>
<dl class="entity_given_name">
<dt>First name</dt>
<dd>
<span class="given_name">Bob</span>
</dd>
</dl>
<dl class="entity_family_name">
<dt>Family name</dt>
<dd>
<span class="family_name">Exampleman</span>
</dd>
</dl>
<dl class="entity_fn">
<dt>Full name</dt>
<dd>
<span class="fn">Bob Exampleman</span>
</dd>
</dl>
<dl class="entity_url">
<dt>URL</dt>
<dd>
<a class="url" href="http://bob.diaspora.example.com/" id="pod_location" rel="me">http://bob.diaspora.example.com/</a>
</dd>
</dl>
<dl class="entity_photo">
<dt>Photo</dt>
<dd>
<img class="photo avatar" height="300px" src="http://bob.diaspora.example.com/uploads/images/thumb_large_sTBJOBJScE.jpg" width="300px">
</dd>
</dl>
<dl class="entity_photo_medium">
<dt>Photo</dt>
<dd> 
<img class="photo avatar" height="100px" src="http://bob.diaspora.example.com/uploads/images/thumb_medium_sTBJOBJScE.jpg" width="100px">
</dd>
</dl>
<dl class="entity_photo_small">
<dt>Photo</dt>
<dd>
<img class="photo avatar" height="50px" src="http://bob.diaspora.example.com/uploads/images/thumb_small_sTBJOBJScE.jpg" width="50px">
</dd>
</dl>
<dl class="entity_searchable">
<dt>Searchable</dt>
<dd>
<span class="searchable">true</span>
</dd>
</dl>
</div>
</div>
</div>

Seed Location

<Link rel="http://joindiaspora.com/seed_location" type="text/html" href="http://bob.diaspora.example.com/"/>

The “seed_location” is a link to bob’s pod.

guid

<Link rel="http://joindiaspora.com/guid" type="text/html" href="((guid))"/>

This is just bob’s guid. When a user creates an account on a pod, the pod MUST assign them a guid – a random hexadecimal string of at least 8 hexadecimal digits.

Activity Stream URL

<Link rel="http://schemas.google.com/g/2010#updates-from" type="application/atom+xml" href="http://bob.diaspora.example.com/public/bob.atom"/>

This atom feed is an Activity Stream of bob’s public posts. Diaspora pods SHOULD publish an Activity Stream of public posts, but there is currently no requirement to be able to read Activity Streams. For more information, read the Activity Streams specification

Note that this feed MAY also be made available through the PubSubHubbub mechanism by supplying a <link rel="hub"> in the atom feed itself.

Diaspora Public Key

<Link rel="diaspora-public-key" type="RSA" href="((base64-encoded representation of the rsa public key))"/>

When a user is created on the pod, the pod MUST generate a pgp keypair for them. This key is used for signing messages. Here we place the key in the “href”. The format of the key is this: We take the ascii-armored representation of the key, and base64-encode THAT. Thus, the actual binary key is “double-wrapped” in base64-encoding. Removing the outer wrapper provides a familiar DER-encoded PKCS#1 key beginning with the text “—-BEGIN RSA PUBLIC KEY—-”. Some platforms may require conversion of this key to a different format, such as PKCS#8 or “modulus/exponent”.

Sending

Note:

This section refers to an new version of federation protocol which may be unavailable on some pods in the federation. As this is changed the note will be removed. Legacy version of the protocol may be found here.

If you (Alice) have decided to share with a user (Bob) on another pod, then you will need to send posts as salmon to the remote user.

You have three tasks:

1. Construct your message.
2. Construct the url for Bob’s salmon endpoint.
3. Post the message to Bob.

In this example, Alice Exampleman (alice@alice.example.com) is attempting to send a message to Bob Exampleman (bob@bob.example.com).

Constructing the message

In Diaspora, there are two types of messages: private and public.

Private messages are sent to remote users encrypted. This helps protect the privacy of your messages while they are in transit, even if you post the salmon to Bob’s pod using regular HTTP instead of SSL-encrypted HTTP. (Note that messages are only guaranteed to be encrypted in transit. They MAY be decrypted by Bob’s server and stored in cleartext).

Public messages conform to Salmon magic envelope specification. To support the encryption semantics for private messages, Diaspora wraps the Salmon magic envelopes into a simple JSON structure.

So, in order to construct the full salmon slap, you will need to:

1. Prepare the payload message.
2. Construct a Diaspora salmon magic-envelope.
3. For a private message you additionally have to wrap the magic envelope

Prepare the payload message

Payload may contain these sorts of things:

• Notification that you’ve begun sharing with them.
• Posts that you’ve made.
• Comments that have been made (by you or others) on one of your posts.
• “Like”s that have been made (by you or others) on one of your posts.
• Conversations (each thread in the inbox has an object representing it)
• Messages (each individual message in a Conversation)
• Profile information
• Retractions of posts
• Retractions of likes/comments

The post content depends on what type of message you are sending. In general, the API is in flux, so this document may be out of date. The authoritative source for this information is the “entities” of the diaspora_federation gem implementation of the federation protocol.

Legacy version of the protocol required to have a wrapping around the post content of the following form:

<XML>
  <post>((post-content))</post>
</XML>

However, the new versions of diaspora* doesn't require it anymore, and support for this wrapping will be dropped in favor of using unwrapped ((post-content)).

The payload must be then base64-encoded.

Construct a salmon magic envelope

You must now construct the salmon magic envelope that we will post to Bob, the recipient. It is constructed thusly:

<?xml version='1.0' encoding='UTF-8'?>
<me:env xmlns:me="http://salmon-protocol.org/ns/magic-env">
  <me:encoding>base64url</me:encoding>
  <me:alg>RSA-SHA256</me:alg>
  <me:data type="application/xml">((base64url-encoded payload message))</me:data>
  <me:sig>((the RSA-SHA256 signature of the above data))</me:sig>
  <me:key_id>((base64url-encoded Alice's diaspora ID))</me:key_id>
</me:env>

Note that the last step in the preparation of the payload message was to base64-encode it. That string must be base64-encoded again to form the <me:data> element. However, this time, it must be encoded with the slightly-different base64url encoding. So your payload message will end up double-wrapped in base64-encoding.

The signature (<me:sig> element) is constructed as specified in the Magic Envelopes specification. That is, use the RSA-SHA256 algorithm to sign the base string with your (Alice’s) private RSA key.

To construct the base string, concatenate the following elements, separated by periods (.).

1. The contents of the <me:data> field. That is the base64url-encoded prepared payload message (remember, the original payload message has now been base64-encoded twice. Once with regular base64, and once with base64url). Note: In previous versions this was then required to be re-padded with linefeeds. This should no longer be done. This is now the actual contents of the <me:data> field. As-is.
2. The base64url-encoding of the “data-type” parameter. In this case, ‘application/atom’ is base64-encoded. Thus, the base64url-encoded string is YXBwbGljYXRpb24veG1s Note: Linefeed character should no longer be included
3. The base64url-encoding of the “encoding” parameter, which is the literal string base64url, base64-encoded. Thus, the base64url-encoded string is YmFzZTY0dXJs Note: Linefeed character should no longer be included
4. The base64url-encoding of the “alg” parameter, which is the literal string RSA-SHA256, base64-encoded. Thus, the base64url-encoded string is UlNBLVNIQTI1Ng== Note: Linefeed character should no longer be included

Sign the base string with your (Alice’s) private RSA key and base64url-encode the results.

This is the final form of the salmon slap for a public message, ready for delivery. If you want to send an encrypted message, proceed to #Wrapping the magic envelope. Otherwise, skip the section and go further.

Wrapping the magic envelope

Choose an AES key and initialization vector, suitable for the aes-256-cbc cipher.

Construct the following JSON object, which shall be referred to as the “aes key bundle”:

{
  "iv": ((base64-encoded AES iv)),
  "key": ((base64-encoded AES key))
}

Encrypt the “aes key bundle” with Bob’s RSA public key. I shall refer to this as the “encrypted aes key bundle”.

Construct the following JSON object, which I shall refer to as the “encrypted wrapper json object”:

{
  "aes_key": ((base64-encoded encrypted aes key bundle)),
  "encrypted_magic_envelope": ((base64-encoded encrypted magic_envelope from above))
}

The encrypted_magic_envelope is the magic envelope encrypted with the aes_key and base64 encoded. After decoding and decrypting it can be parsed like a normal magic envelope (see public messages).

The “encrypted wrapper json object” is what you then post as a private message to the endpoint.

Construct the URL Salmon endpoint

For public messages you use the common endpoint <pod_url>/receive/public.

For private messages to construct the url of the Bob's salmon endpoint, do the following:

1. Get the pod location of the remote user, Bob.
2. Get the guid of the remote user (using the webfinger process described above).
3. Construct <pod_url>/receive/users/<guid>. This Bob’s salmon endpoint.

Post the message to Bob

Take your final delivery package, double urlencode it, and POST this data to the salmon endpoint. For public messages you must use Content-Type: application/magic-envelope+xml and for private messages Content-Type: application/json.

If you receive an HTTP 202 Created, or a 200 OK, your package has been accepted.

Receiving

Consider the case in which you are Bob, receiving a salmon slap from Alice. In general, you should be able to follow the steps outlined in the “Sending” section, in reverse. Verify the slap, as specified in Section 8 of the Salmon specification, and return 202 Created if this is a new salmon, or 200 OK if this salmon updates a previous one. If the slap fails verification, return 400 Bad Request.

Note that the slap sent to Bob is signed with Alice’s private key. However, nothing about the author of a slap is sent in cleartext. Therefore, you will have to decrypt and decode the payload message in order to find the information about who the message is from. The payload message will contain the diaspora handle of the author. Get their public key from their webfinger protocol. This is the key that you will use to verify the signature.

When verifying the signature, note that the reference implementation of Diaspora uses the Ruby OpenSSL library, which generates RSA keys in the PKCS#1 format. Another popular key format is the new PKCS#8 format. Some tools may not interoperate. For example, the openssl command-line tool has options to verify RSA signatures, but it can only read keys in the PKCS#8 format. To complicate matters, there is no header information that says which of these formats the key is written in, so the openssl command-line tool cannot return a more informative error than “unable to load Public Key”.

For more information on this problem, see this blog post.

Additional Diaspora Protocols

Out of date:

This section's accuracy may be compromised due to out-of-date information. Please help improve the page by updating it. There may be additional information on the talk page.

Diaspora pods MAY offer federation through other protocols as well. The reference implementation offers the following additional protocols:

• An ActivityStream of public posts.
• An API that allows third-party applications to use your pod’s data on behalf of your users, using OAuth authentication (the OAuth protocol).

ActivityStream of public posts

The reference implementaion of Diaspora exposes a UI that allows users to mark posts as “public”. If Alice makes a post that is not marked as public, the post will be sent only to those people that Alice is sharing with. However, if Alice makes a post that is marked public, it will also be sent to those people that are sharing with Alice, even if Alice is not sharing with them.

In addition, the posts are added to an ActivityStream. The address of this feed is published in the user’s hcard, with:

<Link rel="http://schemas.google.com/g/2010#updates-from" type="application/atom+xml" href="https://joindiaspora.com/public/((username)).atom"/>

The feed SHOULD also be made available on a PubSubHubbub server.

See the ActivityStrems specification and the PubSubHubbub specification.

Third-party Application API

The reference implementation of Diaspora offers an API for third-party application developers. It allows third-party applications to use your pod’s data on behalf of your users. Access is controlled by the the OAuth protocol.

Right now, there is only one application that uses this API. Cubbi.es. The API is still very much in flux, so people are not being encouraged to write new applications until the protocol has been solidified a little more.