RLN in the Browser

[fryorcraken]

Meta issue: vacp2p/research#129

Breakdown

We want (1) , (2) and (3) for DevCon. Order on the other user stories in loose.

1. As a user, I want to join the group

• Generate RLN credentials
  • Integrate zerokit in js-rln
• Call the contract to join the group
  • Using web wallet

2. As a user, I want to send a message that includes a proof

Would probably use Light Push (but does not really matter)

• Retrieve state of group (must be latest to be secured)
  • Call contract (every time?)
  • Listen to contract events
• Generate the proof using zerokit
• Attach the proof to Waku Message
  • Need to design API/interface so messages are attached every time relay.send or lightpush.push is used.

3. As a end user, I want to use RLN in the web

• Create rln-web-chat (clone web-chat) New example https://examples.waku.org/rln-js/ https://github.com/waku-org/js-waku-examples/tree/master/rln-js

4. As a rln-js user, I want to my RLN credentials to be securely saved in the browser storage

• Credentials are saved in the browser

• Credentials derived from Ethereum Wallet signature

• This is a nice feature for web-chat users

• Also an example for developers

• See how eth-pm saves encryption key

5. As a rln-js user, I want to be able to export and import my RLN credentials

• Can import text credentials
• Can export credentials by copy-pasting
• Can export/import credentials in file format, blocked by 17/WAKU-RLN-RELAY: File format to backup credentials? vacp2p/rfc#543

6. As a user, I want my node to check proofs on messages

• Expose interface to verify proof
• Provide an easy API to automatically drop messages with invalid proofs
• Provide an easy API to track seen epochs

The message could be received via store, filter or relay.

Note: This could be pass as a generalized "validation callback" method to filter and store. Maybe some helper to avoid too much code.

7. As a user, I want Waku Relay to check proofs on messages and drop them if invalid

Useful once Relay scalability #905 is ready.
Preferably use pubsub validator.

8. As a user, I want to be able to slash spammers

Review if slashing makes sense in the browser once #905 is progressed.

Other requirements

• create js-rln repo under waku-org
• Use Apache 2.0 - MIT Licence
• Use similar tech stake than js-waku:
  • language: TypeScript
  • ESM Only package
  • package manager: npm or pnpm
  • Docs: Typedoc
  • TypeScript to JavaScript: tsc or parcel or vite
  • Bundling (to enable direct usage in <script> tag): rollup or vite
  • Format: prettier + eslint
  • Testing:
    • mocha/chai is used for js-waku, ok to consider more modern alternative, consider aegir
    • Browser run: karma, ok to consider better alternative or aegir (js-waku uses webpack for karma, best to try rollup or whichever bundler is used for prod code).
    • NodeJS: 16 or 18 (js-waku uses 16, tests can't run with 18 due to ESM loader changes, fine to use 18 and backport config fix to js-waku)
  • Dependencies: aim to use similar deps to js-waku for similar purposes (ie, noble, etc). Avoid polyfills.

[fryorcraken]

2 Sep

Attendees

• @staheri14
• @richard-ramos
• @fryorcraken

Minutes

JS-RLN Status

• @richard-ramos has started the work in getting zerokit working in JavaScript:
  • compiling zerokit in wasm
  • Using wasm API in JavaScript
  • Generate witness
  • Generate proof
  • Verify proof

Issue encountered: a wasm function cannot load a wasm payload of more than 4.0KB.
Worked around by loading the payload in JS and passing it to the wasm API.

The PoC is local JS code in one file. Next steps:

• Verify proof
• Tidy up the code
• Move the code to js-rln repo, use TypeScript

JS-RLN RAID

Merkle tree size download

It was discussed whether a browser can handle 60-100MB in memory. From testing, this is possible.

However, downloading 60MB/100MB would be a UX issue:

• data plan restriction
• download speed (could take several minutes)

Hence, we may need to have strategies in place such as:

• Light mode: Trust a remote node to provide the minimum information required to generate proof (similar how a remote is trusted to deliver a message when using light push)
• Heavy mode: Trustless setup (proof is downloaded) but this comes with UX drawbacks. This could also be done with a browser extension that starts downloading the merkle tree when the browser is started. (similar to Waku Relay scability [Epic] Waku Relay scalability in the Browser js-waku#905)
• Hybrid mode: Light mode at first while merkle tree is downloaded in the background.

Do note that for now the Merkle tree is few kB so it's not an issue just yet.

It was discussed that the size of the tree comes to the fact that we need to be able to delete members.

Another note is that to download the Merkle tree from the smart contract, the stack involves several components:

• etherjs library
• Browser Wallet Extension (e.g. MetaMask) or dApp browser (e.g. Status)
• Web3 API provider (e.g. infura)

We may face issue with any of this step due to data size limits or timeout due to the length of time taken to download the data.

Usage of zerokit in go-waku

@richard-ramos did some build experiments with zerokit (go-waku currently uses kilic/rln) and the static library results are 100s of MB large: vacp2p/zerokit#37.

This is not ideal for mobile environment (large app) and can be blocker for Waku React Native (uses go-waku) as there are timeout and size limits to upload packages to npmjs.com (that's how waku react native is currently distributed).

@staheri14 mentioned that there are two versions of the zerokit library in the repo, with one being lighter.

• @richard-ramos to check with @s1fr0

[staheri14]

@staheri14 mentioned that there are two versions of the zerokit library in the repo, with one being lighter.

Not sure about this tbh, my point was about the tree size, which may/may not have an impact on the lib size, @s1fr0 can confirm better.

[staheri14]

Related to the tree size, fyi, we have published a forum post covering some initial analysis of the storage overhead of the membership Merkle tree https://forum.vac.dev/t/waku-rln-relay-evaluating-storage-overhead-of-membership-merkle-tree/151
Feel free to comment and share concerns

[oskarth]

compiling zerokit in wasm

Why are we doing this? Seems like a wasted effort for no reason. Why are we not using the existing JS implementation of RLN that works? cc @staheri14

[richard-ramos]

:( I did not know about the existence of https://github.com/njofce/zk-chat/blob/main/zk-chat-client-lib/

[fryorcraken]

In regards of using zerokit (wasm) vs the JS library used by zk-chat

Zerokit

Benefits

• Long term, better to use a library that we are maintainers of so that we can better influence the quality and features present in the library
• One Rust library to serve all implementations (nim, go, js) reduces maintenance, research & development effort
  • This includes decreased chance of interoperabiity issues, or delta in the features/development available across platforms

Risks

• The current size of the wasm blobs are ~7MB (5.7MB for the circom circuit, 838kB for zerokit), this is too large to build a responsive Web app (should target 100kb or under).
• Circuit: currently passed base64 encoded, we could reduce the size by trying different encoding and zipping. Actual size are:
  • 3.1M rln_final.zkey (gzip'd 2.2MB)
  • 1.2M rln.wasm (gzip'd 561kB)
• Circom: @s1fr0 has already done work to reduce the shared library object size (by using different/smaller dependencies), there is chance that this could impact the wasm size

In summary, we don't know if it's possible to reach an ideal library size for the web, and how effort would be involved (e.g. Forking dependencies, upstream feature flags, more library split, etc).

zk-kit

Benefits

• Smaller library size: unsure, npm displays
  • https://www.npmjs.com/package/@zk-kit/protocols 55kB
  • https://www.npmjs.com/package/@zk-kit/identity 24.6kB (but readme shows 1.59MB)
  • The full client: https://www.npmjs.com/package/zk-chat-client 282kB

But, zk-kit does not provide the circom circuit, so we would still need to pass ours or find theirs.
The developers using zk-kit are supposed to provide the

Cons

• Uncertainty if there are differences with zerokit and how would we manage them (upstream feature flags? for repo?)
• The risk around circom size remain. Also, if their circom circuit is smaller then can we replicate the technique to make it smaller on our circuit?
• https://github.com/njofce/zk-chat uses a client/server model (see zk-chat-server-lib), whie we would use the @zk-kit libraries, it does mean that there some unknowns unknowns here (does some of the code work only in NodeJS? etc) as they do not run the JS code only in the browser.

Conclusion

Using zerokit does involve some risk and unknown unknowns around being able to reduce library/circuit size to an idea web app size.

However, the risk still partially exists with zk-kit.

Added maintainability risks on using a 3rd's party library makes zerokit a safer choice.

Follow-up

Are there other libraries available?

[fryorcraken]

Other project:
https://github.com/Rate-Limiting-Nullifier/zk-keeper

uses the same library than zk-chat actually:

"@zk-kit/identity": "^1.4.1",
"@zk-kit/protocols": "^1.8.2",
"circomlibjs": "^0.0.8",

Not sure why they would need circomlibjs. Circuit generation on the fly? Maybe something to check out @richard-ramos as circom circuit size is one of the challenges.

Edit: circomlibjs is actually 9MB so not really an option for a webapp (makes sense in an extension).

The other interesting part is whether the extension is generic enough so it could be use for RLN.

[oskarth]

@fryorcraken Thanks for the breakdown from your POV! This is exactly the kind of discussion we should've had at the beginning of this work, but glad we are doing it properly now! Would like to hear @richard-ramos and @staheri14 thoughts on this too.

But, zk-kit does not provide the circom circuit, so we would still need to pass ours or find theirs.

We are using the same and they are here: https://github.com/privacy-scaling-explorations/rln

Uncertainty if there are differences with zerokit and how would we manage them

My intuition is that they are not that big (especially compared to initial WASM blocker, which might be addressed now?), but I could be wrong and to give a qualified answer you'd have to look at RLN Relay API, MT algo etc and do a more thorough diff, something I think @richard-ramos @staheri14 @staheri14 are able to do.

The risk around circom size remain. Also, if their circom circuit is smaller then can we replicate the technique to make it smaller on our circuit?

That's not a risk relevant to zk-kit, it is the same for both.

https://github.com/njofce/zk-chat uses a client/server model (see zk-chat-server-lib), whie we would use the @zk-kit libraries, it does mean that there some unknowns unknowns here (does some of the code work only in NodeJS? etc) as they do not run the JS code only in the browser.

While this is true and it is a different model, IIRC there's no Node specific code that does the core RLN operations we care about. Could be wrong though, so requires looking a bit more.

---

Long term I agree we probably want to use Zerokit for everything, if possible. The most immediate goal now is to get this working end-to-end for testnet2 and Devcon, so the question right now is what would be the fastest reasonable path towards that? What do you think @richard-ramos

[staheri14]

It was non-trivial to find the commit version of the zk-kit repo in which the rln module existed (it is now deleted from the zk-kit master branch), but finally found it:

• The implementation of the rln part https://github.com/privacy-scaling-explorations/zk-kit/blob/9c8c2723229ef7ea9e1b90568efbf2622c2b67fa/packages/protocols/src/rln.ts
• The corresponding tests https://github.com/privacy-scaling-explorations/zk-kit/blob/9c8c2723229ef7ea9e1b90568efbf2622c2b67fa/packages/protocols/tests/rln.test.ts#L31
• The wasm files reside in https://github.com/privacy-scaling-explorations/zk-kit/blob/fdede4d9b587c0aa8c895f17b700b6c08f06f766/packages/protocols/zkeyFiles.zip (size is 4.95 MB)

My thoughts and findings so far:
Their public and private inputs for the rln proofs LOOK SIMILAR to the zero-kit (but not killic library). zk-kit/rln seems to support secret sharing, and slashing, and also utilize a rlnIdentifier for secret sharing, which matches the vac/zero-kit. But we need to have the circuit file to be sure about the underlying computations.

• The rln-related APIs of zk-kit differ from zero-kit, so I need to play more with both repos to figure out the mapping between the two, the data sterilization/deserialization, and their parameters, etc.
• zk-kit/rln uses the same curve as zero-kit, bn128 .
• The zk-kit/rln witness generation seems inefficient (see generateMerkleProof function call and the actual implementation). For EACH rln witness generation, the entire tree needs to be reconstructed from the set of leaves and then the MerkleProof object is extracted from that tree. It is perceivable that this would slow down the proof generation a lot. In zero-kit, the tree is constructed once and is used for the subsequent rln witness generations.
• Regarding the item above, my understanding is that we might be able to benefit from the IncrementalMerkleTree class in the same repo to construct the tree once and then use createProof function to populate the required MerkleProof object. In a nutshell, it means the tree gets created once and used for all the rln witness generations, similar to zero-kit.
• The IncrementalMerkleTree class is implemented in such a way that it holds the entire tree, similar to zero-kit, hence supports both member addition and deletion.

ATM, there is no absolute answer re whether the two libs are the same or not, the more time we invest the more certainty we get about this. I am still investigating.

cc: @fryorcraken @richard-ramos @oskarth

[staheri14]

Apparently, the rln relevant functionality is moved from zk-kit repo to this repo https://github.com/Rate-Limiting-Nullifier/rlnjs/

[fryorcraken]

16 Sep - @staheri14 / @fryorcraken catch-up

• Note that RLN in the Browser #1 (comment) was written with @richard-ramos so I believe we are on the same page

• Will cancel this meeting to attend Monday/Tuesday Vac PM instead

• Regarding zerokit risks: The main risk the bundle size, see RLN in the Browser #1 (comment) for breakdown

  • 838kB library: Note that js-waku was 1.1MB 2 months ago, 838kB is manageable for a demo/PoC
  • Circuit size: Using binary blob (instead of base64) and gzip enables a reduction by 50%. We can play around with other method. Also need to clarify if this asset download can be done separately so it's done while the user connect their wallets, etc.

• Hence, for now let's keep focus on zerokit. No need to spend more time investigating difference with rlnjs (@richard-ramos, do you agree?)

• rln-js update:

  • @fryorcraken had to prepare js-waku to enable RLN to be plugged in. Spend the past week on this. The change is almost ready: Make Waku Message Modular js-waku#935
    • Had to do other refactoring, which are now merged and released
    • Had to ensure examples are up to date, especially web-chat
    • Next steps:
      • finish update of examples with latest js-waku release
      • Wrap up & mergehttps://github.com/Make Waku Message Modular js-waku#935 (rebase + some minor changes)
      • Integrate these changes in examples
  • @richard-ramos to implement the new js-waku's Encoder interface in RLN to append the proof to the proto message.

Then, @fryorcraken will clone web-chat and start integration of Web wallet.
Risks: web-chat uses a React framework for the chat component, uncertain how to fit new buttons/options with this framework.
Currently thinking about a sidebar with the following options:

1. Connect Wallet Button
2. Sign transaction to send to RLN contract (contains RLN credentials) Button
3. Display RLN credentials
4. Button to export RLN credentials
5. Form to import RLN credentials

[richard-ramos]

@oskarth: The most immediate goal now is to get this working end-to-end for testnet2 and Devcon, so the question right now is what would be the fastest reasonable path towards that? What do you think @richard-ramos

After discussing this with @fryorcraken, we concluded that continuing the zerokit/wasm approach is the path we should take, since in practice most of the work required in js-rln for usage in js-waku is complete (besides bugs that might be reported in the PR and the size of zerokit output .wasm file).

[fryorcraken]

When starting on export/import, best to talk with zk-keepers team.

[fryorcraken]

FYI @oskarth @staheri14 I am not sure we'll have time to integrate it in web-chat.

Instead, we are focusing on examples.waku.org/rln-js/

This will still demonstrate sending messages with proof, receiving messages with proof and verifying them

It's a bit more raw. The upside is that it's just javascript and HTML in a single page, hence more readable to any developer than if it was a React app

[fryorcraken]

Devcon deliverable (and more!) achieved:

• Website: https://examples.waku.org/rln-js/
• Code: https://github.com/waku-org/js-waku-examples/tree/master/rln-js
• Video: https://drive.proton.me/urls/EC4G8SY2J8#ie92Wtje1f4O

[fryorcraken]

Latest demos:

• https://www.youtube.com/watch?v=Xz5q2ZhkFYs
• https://www.youtube.com/watch?v=-vVrJWW0fls

[fryorcraken]

Need to tidy up the issue, close it and open new issues for follow up if needed.

[fryorcraken]

I suggest to close this for now:

5. As a rln-js user, I want to be able to export and import my RLN credentials

New issue to be opened once vacp2p/rfc#543 is ready

6. As a user, I want my node to check proofs on messages

Interface exposed to check proof. Other API improvement can be tracked as we push for adoption and look at improving devex

7. As a user, I want Waku Relay to check proofs on messages and drop them if invalid

For now light client protocols are recommended from the browser, we can leave relay validation to nwaku nodes

8. As a user, I want to be able to slash spammers

Slashing is not ready, new issue can be open if and wehn we want to be able to slash from the browser.