SeM(Milestone): Scaling Status App to 1 million users MVP

[kaiserd]

This milestone issue captures SeM tasks for the Status 1 million users MVP.
It stretches over several SeM tracks, among them Secure Scaling, Application Protocols, Discovery, and Data Sync.

This issue is related to the Waku 10k milestone.

Waku Relay Scaling Plan

The general task for SeM is to come up with a simple scaling strategy that allows scaling a Waku network to 1 million users.
The basis of our idea is sharding the Waku relay network, i.e. using several pubsub meshes instead of a single one.
To keep up with the tight timeline for this MVP, trade-offs to privacy&anonymity have to be made.

Our goal is achieving 10k users in a single shard and scaling the network to 1 million users using multiple shards.
In the context of this milestone, this limits the size of a single multicast group (and by extension the size of a single community) to 10k active users.

SeM work towards the MVP is split into tree main tasks

1. scaling a single shard (pubsub mesh) to 10k users.
   Our simple model we developed as part of this scaling plan showed:
   current Waku relay should theoretically already be able to scale to 10k nodes per shard.
   Our main focus here is identifying practical issues as well as problems arising from the way Status currently uses Waku Relay.

2. scaling the Waku network used by Status App to 1 million users (assuming we solved 1).
   Our focus here is coming up an easy-to-deploy approach that allows discovering nodes associated with a subset of a set of ~1000 shards.
   We need at least 100 shards for communities and another 100 shards for 1:1 chats.
   Here is the current version of our relay sharing RFC.
   For the Status 1 million users MVP, we will use the static sharding method specified in this RFC.
   Other work here comprises ways of mapping Status communities to shards.

3. specifications for both Status protocols and scaling extensions for Waku protocols.

Regarding the store protocol, we pursue a simple approach for the MVP:
store nodes are federated and provided by the respective community; they cover each content topic within the respective community.
We can also add powerful infrastructure store nodes that cover all pubsub meshes relevant to the Status app.
The work on store towards the MVP is mainly driven by the Waku product team.

owner: @kaiserd

Waku Relay Scalability Analysis

• Analysis: Waku Relay scaling (part 1) #158

• (we might extend the model / analysis based on requirements)

• add model case for community control messages
• add model case for flood-publish (if significant)

owner: @kaiserd

Discovery

We are working on improvements and simplifications of the static sharding method.

• topic sharding improvements

owner: @kaiserd

DoS mitigation

• Simple Waku Relay DoS mitigation #164

owner: @alrevuelta

Communities

• extract how Status communities work as a basis for a specification (from code and asking CCs)
  • main work here is done modulo some clarifications and follow-ups
• identify scaling problems
  • current status: Scaling Status Communities : Potential Problems #177

owner: @rymnc

1:1 chat

• extract how Status 1:1 chat works as a basis for a specification (from code and asking CCs)
  • main work here is done modulo some clarifications and follow-ups
• identify scaling problems

owner: @rymnc

Status App Protocols Mapping

• research for: raw RFC(BCP): Status communities over Waku Simple Scaling #167

owner: @rymnc, @kaiserd

RFCs

• New RFC: first version of Waku Relay Sharding #163
  • (we are still working on improvements here)
• raw RFC(standards): Status communities #166
• raw RFC(standards): Status 1:1 chat #168
• New RFC: BCP Waku Simple Scaling #160
  • raw RFC(BCP): Status communities over Waku Simple Scaling #167
  • Dos Mitigation ( Simple Waku Relay DoS mitigation #164 ) will be part of this RFC

owner: @kaiserd, @rymnc, @alrevuelta

We plan to publish each of these RFCs (at least in raw status) by the end of Q1 2023.

CC @cammellos @felicio @fryorcraken @oskarth @corpetty @John-44

[rymnc]

Additional RFC to be included in this milestone (59/STATUS-URL-DATA) (PR) -

status-im/specs#159 (comment)

[felicio]

Additional RFC to be included in this milestone (59/STATUS-URL-DATA) (PR) -

status-im/specs#159 (comment)

I consolidated the info for you. Feel free to push new commits and make changes as you see fit.

• feat(59/STATUS-URL-DATA): initial draft rfc#600
• feat(60/STATUS-URL-SCHEME): initial draft rfc#602

[kaiserd]

Most tasks in this issue have been completed.
The Waku Milestone: waku-org/pm#83 continues the effort.

The task add model case for community control messages for the Waku scaling analysis is tracked in: the Vac DST roadmap