feat: use metadata protocol for awaiting connection to remote peer

[danisharora099]

Problem

Parent/origin: ##1756

There is a requirement to wait for ~50/100ms after a peer:connect event is fired as the Metadata Protocol is an async operation for which we need to wait to populate shard info, or we are disconnected if there is a cluster ID mismatch.

We need to wait for the Metadata protocol to complete before we can assume the peer is ready to be used to open connections reliably.

Solution

• Add check for successful metadata handshakes in waitForRemotePeer
• Also make fixes around enabling sharding for nwaku nodes (chore: bring clarity for running a node on a cluster-id shard nwaku#2259)

Notes

Can still foresee a case where a peer that is connected is used for a protocol instantly, thus creating a race condition.
Perhaps a more protocol-knit check should be added. Will also be helped with #1463

[github-actions]

size-limit report 📦

Path	Size	Loading time (3g)	Running time (snapdragon)	Total time
Waku core	29.8 KB (+0.12% 🔺)	597 ms (+0.12% 🔺)	212 ms (+61.58% 🔺)	808 ms
Waku Simple Light Node	242.94 KB (+0.16% 🔺)	4.9 s (+0.16% 🔺)	333 ms (+81.62% 🔺)	5.2 s
ECIES encryption	30.24 KB (0%)	605 ms (0%)	162 ms (+86.54% 🔺)	767 ms
Symmetric encryption	30.22 KB (0%)	605 ms (0%)	124 ms (+14.86% 🔺)	728 ms
DNS discovery	121.81 KB (0%)	2.5 s (0%)	215 ms (+63.03% 🔺)	2.7 s
Privacy preserving protocols	118.6 KB (0%)	2.4 s (0%)	216 ms (+34.11% 🔺)	2.6 s
Light protocols	27.45 KB (+0.14% 🔺)	549 ms (+0.14% 🔺)	109 ms (+72.36% 🔺)	658 ms
History retrieval protocols	26.38 KB (+0.11% 🔺)	528 ms (+0.11% 🔺)	80 ms (+48.66% 🔺)	607 ms
Deterministic Message Hashing	5.92 KB (0%)	119 ms (0%)	25 ms (+3.78% 🔺)	143 ms

[fryorcraken]

I would suggest to have an event for the metadata protocol such as "shard info received" and use this as a way to track whether the protocol has been run.

[weboko]

I think this PR should be prefixed with feat!:

[danisharora099]

I think this PR should be prefixed with feat!:

changed to feat -- how do you think this is a breaking change that would require the !?

[danisharora099]

I would suggest to have an event for the metadata protocol such as "shard info received" and use this as a way to track whether the protocol has been run.

This way of checking was used because on the off chance the service node does not send a request to us, or takes more time (which it is gauged by manual tests), we will initiate a request and ensure/enforce that we are on the same shard.
Listening on an event listener can take more time since we are expecting the node to reach out to us.

Let me know if you think otherwise @fryorcraken

[danisharora099]

I would suggest to have an event for the metadata protocol such as "shard info received" and use this as a way to track whether the protocol has been run.

This way of checking was used because on the off chance the service node does not send a request to us, or takes more time (which it is gauged by manual tests), we will initiate a request and ensure/enforce that we are on the same shard. Listening on an event listener can take more time since we are expecting the node to reach out to us.

Let me know if you think otherwise @fryorcraken

Merging this PR for now. Happy to address the proposed change of moving to an event-based functionality as a followup if decided.