Relay servers for public posts

Note:

This is a proposal, not an accepted specification.

Note:

Discussion regarding this on Loomio.

The problem

Diaspora is an excellent communication platform for sharing ideas and discussing them. Decentralization however, while bringing benefits also causes some issues. One of those "make or break" issues is the lack of federation for public posts. Setting up your own pod at the moment doesn't really make sense since you will instantly lose one of the biggest features - tag following. Since posts are generally only delivered to pods where there are participants for that post, lonely pods will not generally get a bulk of the public posts going around.

This creates a broken network and this proposal aims to fix that in a way that in a more lighter way than just pushing all public posts to all pods (which technically is not a good thing to do).

Concepts of the solution

The central hub

We need a central hub. This central hub should be maintained by appointed community members until proper foundation resources can be used. For transparency ALL the data stored on the central hub should be public. The functionality of the central hub would be to hold an official pod list (opt in for podmins of course) and relating to this relay server functionality proposal hold a list of tags each participating pod is subscribing to.

Summa summarum;

• Central hub that is maintained by appointed community members
• ALL data on central hub should be open (REST API)
• Pods add themselves and maintain their data (opt in)
• Pods can also list tags they are interested in

Relay server

A relay server is a lightweight server app that has only one function - passing of messages from one place to another. Technical specs could be, for example (just to not be too Ruby specific :));

• NodeJS
• Redis

There should not be only one relay server. There should be many relay servers and anyone in the community should be able to run one. When someone runs a relay server, that relay will notify the central hub that they are open for business. The central hub in turn notifies pods (when they ask for this) about a list of relays available.

If the relay app could use the normal Diaspora protocol for communication it should.

Events

Someone on a pod adds a hashtag to his/her followed hashtag lists

If pod is not already subscribing to this hashtag, it will notify the central hub that it wants these posts.

Someone on a pod unfollows a hashtag

If no one else on the pod follows this hashtag, pod will notify central hub that it doesn't want these posts any more.

Someone writes a public post with one or more hashtags

If podmin has opted in to relay functionality, pod will randomize one of the cached relays and delivers that post to the designated relay.

Relay receives a post from a pod

Relay already has a cached list of pods and what hashtags they want so relay will deliver post to pods that are interested in one or more of the hashtags in this post. Relay is not for public message keeping - it will delete any posts as soon as they have been pushed out.

Pod receives post from relay

Pod checks from originating pod that the post hash matches the one from the relay server. If hash matches, post is added to pod.

Diagrams

Data flow

Process of post delivery

Alternative solution that doesn't require core code changes

This solution is based on using the current federation protocol and manual user action from users to subscribe to tags. This solution would not require core code changes and could be used as an initial stage. This would not transparently make users receive public posts from the network, it would require each user sending out ```subscribe/unbsubscribe``` messages.

The downside of this solution vs the original one is that each relay would need to keep track of all users subscribing, instead of relays just needing to know which pods subscribe. For privacy, this solution is much worse.

Relay server

Federation

In this solution, the relay server would talk like a normal diaspora* pod - in terms of federation it would look to other pods just like a normal pod. However, it doesn't need to support the whole protocol, only a subset. These parts include;

• Share / stop sharing
• Receive private and public status messages
• Push out private and public status messages

Everything else can be ignored.

Tech stack

The server could be built taking the current diaspora* server federation parts or using some other solution, for example Pyaspora.

Sharing of information between relay servers

Initially, one server could be good enough for POC. The relays should however also exchange information between them via the normal protocol status messages. Each relay would keep track of other relays out there and notify them of changes regarding subscribers. If a relay goes down for too long a time, it is forgotten by all the other relays.

Subscribing and unsubscribing to tags

When a user wants to subscribe to a tag, they first start sharing with a relay of choice. Then once the sharing is complete, they write mention the relay in a status message, listing which tags they would like to subscribe or unsubscribe to. This message is sent as normal to the relay server, which then notes down the changes in its db (and notifies other servers too). For the relay, it really only cares about which pod is interested in which tags - but the way to keep track of this is to work with users directly.

Delivery of public messages

Only messages from users who are sharing with a relay will be delivered to relays, due to the fact how the current code works. Once the relay receives the message, it will look at its db and see which pods are interested in the message, by analysing the tags it contains. It will then deliver the message to the pods who are interested (minus the sending pod of course).

After delivery, it will delete the message. Any additional copies of it will be redelivered (for example a possible future post edit feature).