Federation protocol overview: Difference between revisions

From diaspora* project wiki
(→‎Constructing the message: Changes according to https://github.com/SuperTux88/diaspora_federation/issues/30)
(Redirected page to Federation)
 
(6 intermediate revisions by 3 users not shown)
Line 1: Line 1:
The purpose of this document is to describe the communications that go on between Diaspora servers. Implementers of this protocol should be advised, though, that Diaspora is in Alpha, and as such, this document is not authoritative, and may lag behind the reference implementation. You can find an implementation of this protocol that you can reuse in [https://github.com/SuperTux88/diaspora_federation this gem]
#REDIRECT [[Federation]]
 
= Asymmetric Sharing =
 
One important thing to note is that Diaspora’s notion of “sharing” is asymmetric.
 
In a symmetric “friending” relationship, neither party sees any posts from the other until one party makes a friend request and the other confirms.
 
In an asymmetric “sharing” relationship, if you start sharing with someone else, you have decided to send them posts. They may or may not choose to share with you as well. If they choose not to, you will see only their posts that they have explicitly marked as public.
 
Public posts are sent in salmon, just like all posts.
 
= Core Diaspora Protocols =
 
Diaspora servers communicate with one another in a variety of situations:
 
* When discovering information about users on another server.
* When sending information to people that you’re sharing with. That information includes:
** Notification that you’ve begun sharing with them.
** Posts that you’ve made.
** Comments that have been made (by you or others) on one of your posts.
** “Like”s that have been made (by you or others) on one of your posts.
** Conversations (each thread in the inbox has an object representing it)
** Messages (each individual message in a Conversation)
** Profile information
** Retractions of posts
** Retractions of likes/comments
 
This document does not cover the semantics of each of the messages listed above. For a discussion of these semantics, see [[Federation Message Semantics]].
 
 
== Discovery ==
 
Diaspora pods MUST be able to discover users on other pods, given the other user’s webfinger address. For convenience, Diaspora pods’ user-interfaces MAY choose to allow users to search for users by name, searching through the list of names already known to the pod (such as local users). However, pods’ user interfaces MAY NOT allow users to find a person by name if that person has not marked themselves as “searchable”, in their hcard (see below).
 
If alice@alice.diaspora.example.com wants to discover bob@bob.diaspora.example.com, then alice’s pod must perform a Webfinger lookup of bob’s address. Webfinger is an open protocol. See [http://code.google.com/p/webfinger/wiki/WebFingerProtocol the Webfinger protocol specification] for the full details. However, we will summarize here. Note that bob’s webfinger profile does not need to be hosted by bob’s diaspora pod. Any webfinger server will do, so long as bob’s profile contains the elements necessary for Diaspora. See below. However, Diaspora pods SHOULD host webfinger profiles for their users.
 
Alice’s pod will first get the host-meta file from bob’s webfinger address. The host-meta file is located at https://bob.diaspora.example.com/.well-known/host-meta. In the host-meta file, alice’s pod will find a Link element with <tt>rel="lrdd"</tt>, such as:
 
<syntaxhighlight lang="xml">
<Link rel="lrdd" template="https://bob.diaspora.example.com/?q={uri}"  type="application/xrd+xml" />
</syntaxhighlight>
 
Alice’s pod will now transform bob’s webfinger address and replace {uri} with that. First, bob’s webfinger address is url-encoded. Alice will make a GET request to: https://bob.diaspora.example.com/?q=bob%40bob.diaspora.example.com
 
Bob’s webfinger server (which may be the same as bob’s pod) will respond with bob’s webfinger profile. The webfinger protocol might look something like this:
 
<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="UTF-8"?>
<XRD xmlns="http://docs.oasis-open.org/ns/xri/xrd-1.0">
  <Subject>acct:bob@bob.diaspora.example.com</Subject>
  <Alias>"http://bob.diaspora.example.com/"</Alias>
  <Link rel="http://microformats.org/profile/hcard" type="text/html" href="http://bob.diaspora.example.com/hcard/users/((guid))"/>
  <Link rel="http://joindiaspora.com/seed_location" type="text/html" href="http://bob.diaspora.example.com/"/>
  <Link rel="http://joindiaspora.com/guid" type="text/html" href="((guid))"/>
  <Link rel="http://schemas.google.com/g/2010#updates-from" type="application/atom+xml" href="http://bob.diaspora.example.com/public/bob.atom"/>
  <Link rel="diaspora-public-key" type="RSA" href="((base64-encoded representation of the rsa public key))"/>
</XRD>
</syntaxhighlight>
 
Let’s look at these elements in more depth.
 
=== Subject ===
 
<syntaxhighlight lang="xml">
<Subject>acct:bob@bob.diaspora.example.com</Subject>
</syntaxhighlight>
The Subject element should contain the webfinger address that alice asked for. If it does not, then this webfinger profile MUST be ignored by alice’s pod.
 
=== Alias ===
 
<syntaxhighlight lang="xml">
<Alias>"http://bob.diaspora.example.com/"</Alias>
</syntaxhighlight>
(this seems to be the pod location, but with quotation marks around it. Why?)
 
=== hcard ===
 
<syntaxhighlight lang="xml">
<Link rel="http://microformats.org/profile/hcard" type="text/html" href="http://bob.diaspora.example.com/hcard/users/((guid))"/>
</syntaxhighlight>
Bob’s webfinger profile MUST contain a link to an hcard. The hcard contains personal information such as bob’s full name, a link to bob’s photo, etc. Refer to the [http://microformats.org/profile/hcard%20hcard%20specification hcard specification] for a full discussion on hcard syntax.
 
Like the webfinger profile, the hcard need not be hosted by the Diaspora pod. In fact, it may be in a location that is distinct from both the Diaspora pod and the webfinger host. However, if the hcard is to be hosted by the Diaspora pod, then the url SHOULD be: http://bob.diaspora.example.com/hcard/users/((bobs-guid))
 
When a user creates an account on a pod, the pod MUST assign them a guid – a random hexadecimal string of at least 8 hexadecimal digits. The Diaspora pod SHOULD use that guid to create an hcard url, and SHOULD host users’ information as an hcard at this location.
 
Diaspora adds a field, entity_searchable, which contains an element with class “searchable”, which is either “true” or “false”. If the value is false, then pods MAY NOT allow users to search for bob by name, or through any method other than directly entering bob’s webfinger address.
 
Diaspora also includes a url field with the id of “pod_location”, which links to bob’s diaspora pod.
 
Here is an example of an hcard.
 
<syntaxhighlight lang="html4strict">
<div id="content">
<h1>Bob Exampleman</h1>
<div id="content_inner">
<div class="entity_profile vcard author" id="i">
<h2>User profile</h2>
<dl class="entity_nickname">
<dt>Nickname</dt>
<dd>
<a class="nickname url uid" href="http://bob.diaspora.example.com/" rel="me">Bob Exampleman</a>
</dd>
</dl>
<dl class="entity_given_name">
<dt>First name</dt>
<dd>
<span class="given_name">Bob</span>
</dd>
</dl>
<dl class="entity_family_name">
<dt>Family name</dt>
<dd>
<span class="family_name">Exampleman</span>
</dd>
</dl>
<dl class="entity_fn">
<dt>Full name</dt>
<dd>
<span class="fn">Bob Exampleman</span>
</dd>
</dl>
<dl class="entity_url">
<dt>URL</dt>
<dd>
<a class="url" href="http://bob.diaspora.example.com/" id="pod_location" rel="me">http://bob.diaspora.example.com/</a>
</dd>
</dl>
<dl class="entity_photo">
<dt>Photo</dt>
<dd>
<img class="photo avatar" height="300px" src="http://bob.diaspora.example.com/uploads/images/thumb_large_sTBJOBJScE.jpg" width="300px">
</dd>
</dl>
<dl class="entity_photo_medium">
<dt>Photo</dt>
<dd>
<img class="photo avatar" height="100px" src="http://bob.diaspora.example.com/uploads/images/thumb_medium_sTBJOBJScE.jpg" width="100px">
</dd>
</dl>
<dl class="entity_photo_small">
<dt>Photo</dt>
<dd>
<img class="photo avatar" height="50px" src="http://bob.diaspora.example.com/uploads/images/thumb_small_sTBJOBJScE.jpg" width="50px">
</dd>
</dl>
<dl class="entity_searchable">
<dt>Searchable</dt>
<dd>
<span class="searchable">true</span>
</dd>
</dl>
</div>
</div>
</div>
</syntaxhighlight>
 
=== Seed Location ===
 
<syntaxhighlight lang="xml">
<Link rel="http://joindiaspora.com/seed_location" type="text/html" href="http://bob.diaspora.example.com/"/>
</syntaxhighlight>
The “seed_location” is a link to bob’s pod.
 
=== guid ===
 
<syntaxhighlight lang="xml">
<Link rel="http://joindiaspora.com/guid" type="text/html" href="((guid))"/>
</syntaxhighlight>
 
This is just bob’s guid. When a user creates an account on a pod, the pod MUST assign them a guid – a random hexadecimal string of at least 8 hexadecimal digits.
 
=== Activity Stream URL ===
 
<syntaxhighlight lang="xml">
<Link rel="http://schemas.google.com/g/2010#updates-from" type="application/atom+xml" href="http://bob.diaspora.example.com/public/bob.atom"/>
</syntaxhighlight>
 
This atom feed is an Activity Stream of bob’s public posts. Diaspora pods SHOULD publish an Activity Stream of public posts, but there is currently no requirement to be able to read Activity Streams. For more information, read the [http://activitystrea.ms/ Activity Streams specification]
 
Note that this feed MAY also be made available through the [http://code.google.com/p/pubsubhubbub/ PubSubHubbub mechanism] by supplying a <link rel="hub"> in the atom feed itself.
 
=== Diaspora Public Key ===
 
<syntaxhighlight lang="xml">
<Link rel="diaspora-public-key" type="RSA" href="((base64-encoded representation of the rsa public key))"/>
</syntaxhighlight>
When a user is created on the pod, the pod MUST generate a pgp keypair for them. This key is used for signing messages. Here we place the key in the “href”. The format of the key is this: We take the ascii-armored representation of the key, and base64-encode THAT. Thus, the actual binary key is “double-wrapped” in base64-encoding. Removing the outer wrapper provides a familiar DER-encoded PKCS#1 key beginning with the text “—-BEGIN RSA PUBLIC KEY—-”. Some platforms may require conversion of this key to a different format, such as PKCS#8 or “modulus/exponent”.
 
== Sending ==
 
If you (Alice) have decided to share with a user (Bob) on another pod, then you will need to send posts as salmon to the remote user.
 
You have three tasks:
 
# Construct your message.
# Construct the url for Bob’s salmon endpoint.
# Post the message to Bob.
 
In this example, Alice Exampleman (alice@alice.example.com) is attempting to send a message to Bob Exampleman (bob@bob.example.com).
 
=== Constructing the message ===
 
In Diaspora, there are two types of messages: private and public.
 
Private messages are sent to remote users encrypted. This helps protect the privacy of your messages while they are in transit, even if you post the salmon to Bob’s pod using regular HTTP instead of SSL-encrypted HTTP. (Note that messages are only guaranteed to be encrypted in transit. They MAY be decrypted by Bob’s server and stored in cleartext).
 
Public messages conform to Salmon magic envelope specification. To support the encryption semantics for private messages, Diaspora wraps the Salmon magic envelopes into a simple JSON structure.
 
So, in order to construct the full salmon slap, you will need to:
 
# Prepare the payload message.
# Construct a Diaspora salmon magic-envelope.
# For a private message you additionally have to wrap the magic envelope
 
==== Prepare the payload message ====
Payload may contain these sorts of things:
 
* Notification that you’ve begun sharing with them.
* Posts that you’ve made.
* Comments that have been made (by you or others) on one of your posts.
* “Like”s that have been made (by you or others) on one of your posts.
* Conversations (each thread in the inbox has an object representing it)
* Messages (each individual message in a Conversation)
* Profile information
* Retractions of posts
* Retractions of likes/comments
 
The post content depends on what type of message you are sending. In general, the API is in flux, so this document may be out of date. The authoritative source for this information is the [https://github.com/SuperTux88/diaspora_federation/tree/develop/lib/diaspora_federation/entities “entities”] of the diaspora_federation gem implementation of the federation protocol.
 
Legacy version of the protocol required to have a wrapping around the post content of the following form:
 
<syntaxhighlight lang="xml">
<XML>
  <post>((post-content))</post>
</XML>
</syntaxhighlight>
 
However, the new versions of diaspora* doesn't require it anymore, and support for this wrapping will be dropped in favor of using unwrapped ((post-content)).
 
The payload must be then base64-encoded.
 
==== Construct a salmon magic envelope ====
 
You must now construct the salmon magic envelope that we will post to Bob, the recipient. It is constructed thusly:
 
<syntaxhighlight lang="xml">
<?xml version='1.0' encoding='UTF-8'?>
<me:env xmlns:me="http://salmon-protocol.org/ns/magic-env">
  <me:encoding>base64url</me:encoding>
  <me:alg>RSA-SHA256</me:alg>
  <me:data type="application/xml">((base64url-encoded payload message))</me:data>
  <me:sig>((the RSA-SHA256 signature of the above data))</me:sig>
  <me:key_id>((base64url-encoded Bob's diaspora ID))</me:key_id>
</me:env>
</syntaxhighlight>
 
Note that the last step in the preparation of the payload message was to base64-encode it. That string must be base64-encoded again to form the <tt><me:data></tt> element. However, this time, it must be encoded with the slightly-different base64url encoding. So your payload message will end up double-wrapped in base64-encoding.
 
The signature (<tt><me:sig></tt> element) is constructed as specified in the [http://salmon-protocol.googlecode.com/svn/trunk/draft-panzer-magicsig-01.html Magic Envelopes specification]. That is, use the RSA-SHA256 algorithm to sign the base string with your (Alice’s) private RSA key.
 
To construct the base string, concatenate the following elements, separated by periods (.).
 
# The contents of the <tt><me:data></tt> field. That is the base64url-encoded prepared payload message (remember, the original payload message has now been base64-encoded twice. Once with regular base64, and once with base64url). ''Note: In previous versions this was then required to be re-padded with linefeeds. This should no longer be done. This is now the actual contents of the <tt><me:data></tt> field. As-is.''
# The base64url-encoding of the “data-type” parameter. In this case, ‘application/atom’ is base64-encoded. Thus, the base64url-encoded string is <tt>YXBwbGljYXRpb24veG1s</tt> ''Note: Linefeed character should no longer be included''
# The base64url-encoding of the “encoding” parameter, which is the literal string <tt>base64url</tt>, base64-encoded. Thus, the base64url-encoded string is <tt>YmFzZTY0dXJs</tt> ''Note: Linefeed character should no longer be included''
# The base64url-encoding of the “alg” parameter, which is the literal string <tt>RSA-SHA256</tt>, base64-encoded. Thus, the base64url-encoded string is <tt>UlNBLVNIQTI1Ng==</tt> ''Note: Linefeed character should no longer be included''
 
Sign the base string with your (Alice’s) private RSA key and base64url-encode the results.
 
This is the final form of the salmon slap for a public message, ready for delivery. If you want to send an encrypted message, proceed to [[#Wrapping the magic envelope]]. Otherwise, skip the section and go further.
 
 
==== Wrapping the magic envelope ====
 
Choose an AES key and initialization vector, suitable for the aes-256-cbc cipher.
 
Construct the following JSON object, which shall be referred to as the “aes key bundle”:
 
<syntaxhighlight lang="javascript">
{
  "iv": ((base64-encoded AES iv)),
  "key": ((base64-encoded AES key))
}
</syntaxhighlight>
Encrypt the “aes key bundle” with Bob’s RSA public key. I shall refer to this as the “encrypted aes key bundle”.
 
Construct the following JSON object, which I shall refer to as the “encrypted wrapper json object”:
 
<syntaxhighlight lang="javascript">
{
  "aes_key": ((base64-encoded encrypted aes key bundle)),
  "encrypted_magic_envelope": ((base64-encoded encrypted magic_envelope from above))
}</syntaxhighlight>
 
The encrypted_magic_envelope is the magic envelope encrypted with the aes_key and base64 encoded. After decoding and decrypting it can be parsed like a normal magic envelope (see public messages).
 
=== Construct the URL of Bob’s Salmon endpoint ===
 
To construct the url of the salmon endpoint, do the following:
 
# Get the pod location of the remote user, Bob.
# Get the guid of the remote user (using the webfinger process described above).
# Construct <tt><pod_url>/receive/users/<guid></tt>. This Bob’s salmon endpoint.
 
=== Post the message to Bob ===
 
Take your final salmon slap, double urlencode it, and POST this data to Bob’s salmon endpoint as Content-type: application/x-www-form-urlencoded :
 
<pre>xml=((double urlencoded salmon slap))</pre>
If you receive an HTTP <tt>202 Created</tt>, or a <tt>200 OK</tt>, your salmon slap has been accepted.
 
Note that this differs from the standard Salmon protocol, which specifies that you will post only the non-urlencoded salmon slap, without an <tt>xml=...</tt>
 
== Receiving ==
 
Consider the case in which you are Bob, receiving a salmon slap from Alice. In general, you should be able to follow the steps outlined in the “Sending” section, in reverse. Verify the slap, as specified in [http://salmon-protocol.googlecode.com/svn/trunk/draft-panzer-salmon-00.html#SVR Section 8 of the Salmon specification], and return <tt>202 Created</tt> if this is a new salmon, or <tt>200 OK</tt> if this salmon updates a previous one. If the slap fails verification, return <tt>400 Bad Request</tt>.
 
Note that the slap sent to Bob is signed with Alice’s private key. However, nothing about the author of a slap is sent in cleartext. Therefore, you will have to decrypt and decode the payload message in order to find the information about who the message is from. The payload message will contain the diaspora handle of the author. Get their public key from their webfinger protocol. This is the key that you will use to verify the signature.
 
When verifying the signature, note that the reference implementation of Diaspora uses the Ruby OpenSSL library, which generates RSA keys in the PKCS#1 format. Another popular key format is the new PKCS#8 format. Some tools may not interoperate. For example, the openssl command-line tool has options to verify RSA signatures, but it can only read keys in the PKCS#8 format. To complicate matters, there is no header information that says which of these formats the key is written in, so the openssl command-line tool cannot return a more informative error than “unable to load Public Key”.
 
For more information on this problem, see [http://barelyenough.org/blog/2008/04/fun-with-public-keys/ this blog post].
 
= Additional Diaspora Protocols =
 
{{Out of date|part=section}}
 
Diaspora pods MAY offer federation through other protocols as well. The reference implementation offers the following additional protocols:
 
* An ActivityStream of public posts.
* An API that allows third-party applications to use your pod’s data on behalf of your users, using OAuth authentication ([http://tools.ietf.org/html/rfc5849 the OAuth protocol]).
 
== ActivityStream of public posts ==
 
The reference implementaion of Diaspora exposes a UI that allows users to mark posts as “public”. If Alice makes a post that is not marked as public, the post will be sent only to those people that Alice is sharing with. However, if Alice makes a post that is marked public, it will also be sent to those people that are sharing with Alice, even if Alice is not sharing with them.
 
In addition, the posts are added to an ActivityStream. The address of this feed is published in the user’s hcard, with:
 
<syntaxhighlight lang="xml">
<Link rel="http://schemas.google.com/g/2010#updates-from" type="application/atom+xml" href="https://joindiaspora.com/public/((username)).atom"/>
</syntaxhighlight>
The feed SHOULD also be made available on a PubSubHubbub server.
 
See the [http://activitystrea.ms/ ActivityStrems specification] and the [http://code.google.com/p/pubsubhubbub/ PubSubHubbub specification].
 
== Third-party Application API ==
 
The reference implementation of Diaspora offers an API for third-party application developers. It allows third-party applications to use your pod’s data on behalf of your users. Access is controlled by the [http://tools.ietf.org/html/rfc5849 the OAuth protocol].
 
Right now, there is only one application that uses this API. [http://cubbi.es Cubbi.es]. The API is still very much in flux, so people are not being encouraged to write new applications until the protocol has been solidified a little more.
 
[[Category:Technical]]
[[Category:Federation]]
[[Category:Github transfer done]]

Latest revision as of 01:59, 16 August 2017

Redirect to:

Retrieved from "https://wiki.diasporafoundation.org/wiki/index.php?title=Federation_protocol_overview&oldid=9886"