Relay servers for public posts

Work in progress:

This article is a work in progress, it may contain incomplete or inaccurate information. There may be additional information on the talk page.

Note:

This is a proposal, not an accepted specification.

Note:

Discussion regarding this on Loomio.

The problem

Diaspora is an excellent communication platform for sharing ideas and discussing them. Decentralization however, while bringing benefits also causes some issues. One of those "make or break" issues is the lack of federation for public posts. Setting up your own pod at the moment doesn't really make sense since you will instantly lose one of the biggest features - tag following. Since posts are generally only delivered to pods where there are participants for that post, lonely pods will not generally get a bulk of the public posts going around.

This creates a broken network and this proposal aims to fix that in a way that in a more lighter way than just pushing all public posts to all pods (which technically is not a good thing to do).

In fact, this proposal doesn't limit itself to diaspora*, but would also serve other federated social networks using the same protocol (ie RedMatrix and Friendica).

Concepts of the solution

Relay server

A relay server is a lightweight server app that has only one function - passing of messages from one place to another. In terms of federation message support, the server needs to implement:

/.well-known/host-meta to provide discovery for the special relay handle
WebFinger or parts of it, to allow querying of the special relay handle
hCard for the special relay handle
/receive/public to receive public posts. The relay can and should ignore signature verification of authors which will be done by receiving pods anyway
POST of received messages to /receive/public of other pods

There should not be only one relay server. There should be many relay servers and anyone in the community should be able to run one.

Storage

The relay is not required to store messages for any longer than it needs to send them out. The storage should preferable work in a "first in first out" principle, so something like a list would do well.

Pod configuration and statistics.json / NodeInfo

A pod can publish information about itself in the current statistics.json route, which in the future will be replaced by NodeInfo. The attributes related to relays are as follows, in JSON format (with the first possible value as default):

"relay": {
    "subscribe": (false/true),
    "scope": "(all/tags)",
    "tags": []
}

The pod configuration file should have the following available (defaults):

relay:
  outbound_handle: relay@defaultrelay.tld
  subscribe: false
  scope: all
  tags_source: users/podmin

The tags_source setting governs how tags is populated in to the statistics.json / NodeInfo output.

users -- All tags that active users (6mo) follow will be added to the list
podmin -- List of tags is maintained by podmin in the admin menu (needs db table). In the admin menu to not require constant restarts.

Rationale is that podmins can control whether they want the pod to focus on certain topics or let the users influence the topics.

List of active pods

Where the relay gets the active pods list doesn't matter, there are several possibilities already, including http://the-federation.info, https://diasp.net/active and http://podupti.me. A relay should be flexible in falling back to other lists when needed. A relay should also cache the list of pods internally. A relay should query the podlist on a regular basis, for example daily, so newly registered pods don't have to wait for a long time.

Diagrams

Data flow

Process of post delivery