Skip to content

Commit

Permalink
Add section on buffered receipt handling
Browse files Browse the repository at this point in the history
  • Loading branch information
shreyan-gupta committed Dec 16, 2024
1 parent cf3174f commit c4f5e30
Showing 1 changed file with 17 additions and 1 deletion.
18 changes: 17 additions & 1 deletion neps/nep-0568.md
Original file line number Diff line number Diff line change
Expand Up @@ -307,7 +307,23 @@ Since buffered receipts target external shards, it is fine to assign buffered re

#### Handling buffered receipts that target parent shard

TODO(shreyan)
This scenario is slightly more complex to deal with. At the boundary of resharding, we may have buffered receipts created before the resharding event targeting the parent shard. At the same time, we may also have buffered receipts that are generated after the resharding event that directly target the child shard. The receipts from both parent and child buffered receipts queue need to appropriately sent to the child shard depending on the account_id while respecting the outgoing limits calculated by bandwidth scheduler and congestion control.

The flow of handling buffered receipts before ReshardingV3 is as follows:

1. Calculate `outgoing_limit` for each shard.
2. For each shard, try and forward as many in-order receipts as possible from the buffer while respecting `outgoing_limit`.
3. Apply chunk and `try_forward` newly generated receipts. The newly generated receipts are forwarded if we have enough limit else they are put in the buffered queue.

The solution for ReshardingV3 is to first try draining the parent queue before moving onto draining the child queue. The modified flow would look something like:

1. Calculate `outgoing_limit` for both the child shards using congestion info from parent.
2. Forwarding receipts
* First try forward as many in-order receipts as possible from parent shard buffer. Stop either when we drain the parent buffer or as soon as we exhaust the `outgoing_limit` of either of the children shards.
* Next try forward as many in-order receipts as possible from child shard buffer.
3. Apply chunk and `try_forward` newly generated receipts remains the same.

The minor downside to this approach is that we don't have guarantees between order of receipt generation and order of receipt forwarding, but that's anyway the case today with buffered receipts.

### Congestion Control

Expand Down

0 comments on commit c4f5e30

Please sign in to comment.