Skip to content

Commit

Permalink
Document data flow
Browse files Browse the repository at this point in the history
  • Loading branch information
slowli committed Nov 22, 2023
1 parent 215e021 commit 78d4944
Show file tree
Hide file tree
Showing 2 changed files with 23 additions and 1 deletion.
20 changes: 20 additions & 0 deletions core/lib/zksync_core/src/sync_layer/gossip/buffered/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,11 @@ pub(super) trait ContiguousBlockStore: BlockStore {
async fn schedule_next_block(&self, ctx: &ctx::Ctx, block: &FinalBlock) -> StorageResult<()>;
}

/// In-memory buffer or [`FinalBlock`]s received from peers, but not executed and persisted locally yet.
///
/// Unlike with executed / persisted blocks, there may be gaps between blocks in the buffer.
/// These blocks are shared with peers using the gossip network, but are not persisted and lost
/// on the node restart.
#[derive(Debug)]
struct BlockBuffer {
store_block_number: BlockNumber,
Expand Down Expand Up @@ -117,6 +122,21 @@ pub(super) enum BufferedStorageEvent {
}

/// [`BlockStore`] with an in-memory buffer for pending blocks.
///
/// # Data flow
///
/// The store is plugged into the `SyncBlocks` actor, so that it can receive new blocks
/// from peers over the gossip network and to share blocks with peers. Received blocks are stored
/// in a [`BlockBuffer`]. The `SyncBlocks` actor doesn't guarantee that blocks are received in order,
/// so we have a background task that waits for successive blocks and feeds them to
/// the underlying storage ([`ContiguousBlockStore`]). The underlying storage executes and persists
/// blocks using the state keeper; see [`PostgresBlockStorage`](super::PostgresBlockStorage) for more details.
/// This logic is largely shared with the old syncing logic using JSON-RPC; the only differing part
/// is producing block data.
///
/// Once a block is processed and persisted by the state keeper, it can be removed from the [`BlockBuffer`];
/// we do this in another background task. Removing blocks from the buffer ensures that it doesn't
/// grow infinitely; it also allows to track syncing progress via metrics.
#[derive(Debug)]
pub(super) struct Buffered<T> {
inner: T,
Expand Down
4 changes: 3 additions & 1 deletion core/lib/zksync_core/src/sync_layer/gossip/storage/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,9 @@ impl CursorWithCachedBlock {
}

/// Postgres-based [`BlockStore`] implementation. New blocks are scheduled to be written via
/// [`ContiguousBlockStore`] trait, which internally uses an [`ActionQueueSender`].
/// [`ContiguousBlockStore`] trait, which internally uses an [`ActionQueueSender`] to queue
/// block data (miniblock and L1 batch parameters, transactions) for the state keeper. Block data processing
/// is shared with JSON-RPC-based syncing.
#[derive(Debug)]
pub(super) struct PostgresBlockStorage {
pool: ConnectionPool,
Expand Down

0 comments on commit 78d4944

Please sign in to comment.