Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(network): db executor is removed from NetworkManager and runs in a separate task #2122

Conversation

ShahakShama
Copy link
Contributor

@ShahakShama ShahakShama commented Jun 19, 2024

  • refactor(network): remove Data from db executor
  • refactor(network): db executor communicates through channels
  • refactor(network): db executor is removed from NetworkManager and runs in a separate task

This change is Reviewable

@ShahakShama ShahakShama changed the base branch from main to shahak/db_executor_channels June 19, 2024 03:41
Copy link

codecov bot commented Jun 19, 2024

Codecov Report

Attention: Patch coverage is 24.24242% with 50 lines in your changes missing coverage. Please review.

Project coverage is 64.31%. Comparing base (bb9be89) to head (161c1cc).

Current head 161c1cc differs from pull request most recent head a736497

Please upload reports for the commit a736497 to get more accurate results.

Files Patch % Lines
crates/papyrus_network/src/network_manager/mod.rs 0.00% 27 Missing ⚠️
crates/papyrus_node/src/main.rs 28.12% 22 Missing and 1 partial ⚠️
Additional details and impacted files
@@                       Coverage Diff                       @@
##           shahak/db_executor_channels    #2122      +/-   ##
===============================================================
- Coverage                        64.37%   64.31%   -0.07%     
===============================================================
  Files                              136      136              
  Lines                            17941    17900      -41     
  Branches                         17941    17900      -41     
===============================================================
- Hits                             11550    11512      -38     
  Misses                            5113     5113              
+ Partials                          1278     1275       -3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ShahakShama ShahakShama force-pushed the shahak/register_db_executor_through_sqmr_subscriber branch 2 times, most recently from 761368c to cf5b2c9 Compare June 19, 2024 06:57
Copy link
Contributor

@eitanm-starkware eitanm-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 7 of 7 files at r1, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @ShahakShama)

@ShahakShama ShahakShama force-pushed the shahak/register_db_executor_through_sqmr_subscriber branch from cf5b2c9 to 9f874f3 Compare June 19, 2024 10:56
@ShahakShama ShahakShama force-pushed the shahak/register_db_executor_through_sqmr_subscriber branch 3 times, most recently from 19c549c to c75a7c7 Compare June 20, 2024 05:21
Copy link
Contributor

@matan-starkware matan-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 4 of 7 files at r1, 1 of 2 files at r2, 2 of 2 files at r3, all commit messages.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @ShahakShama)


crates/papyrus_network/src/db_executor/mod.rs line 79 at r3 (raw file):

    StateDiffResponsesSender:
        Sink<DataOrFin<StateDiffChunk>, Error = SendError> + Unpin + Send + 'static,
{

These go over 100 columns. Should we use subtypes type Y = XXX

Code quote:

where
    HeaderQueryReceiver: Stream<Item = (Result<HeaderQuery, ProtobufConversionError>, HeaderResponsesSender)>
        + Unpin,
    HeaderResponsesSender:
        Sink<DataOrFin<SignedBlockHeader>, Error = SendError> + Unpin + Send + 'static,
    StateDiffQueryReceiver: Stream<Item = (Result<StateDiffQuery, ProtobufConversionError>, StateDiffResponsesSender)>
        + Unpin,
    StateDiffResponsesSender:
        Sink<DataOrFin<StateDiffChunk>, Error = SendError> + Unpin + Send + 'static,
{

crates/papyrus_network/src/db_executor/test.rs line 171 at r3 (raw file):

            Sender<DataOrFin<StateDiffChunk>>,
        )>,
    >,

type TestDbExecutor = ...

Code quote:

    DBExecutor<
        Receiver<(
            Result<HeaderQuery, ProtobufConversionError>,
            Sender<DataOrFin<SignedBlockHeader>>,
        )>,
        Receiver<(
            Result<StateDiffQuery, ProtobufConversionError>,
            Sender<DataOrFin<StateDiffChunk>>,
        )>,
    >,

crates/papyrus_node/src/main.rs line 143 at r3 (raw file):

    .await?;

    // P2P Sync Server task.

Why do you specify p2p?

Code quote:

    // P2P Sync Server task.

Copy link
Contributor Author

@ShahakShama ShahakShama left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @matan-starkware)


crates/papyrus_network/src/db_executor/mod.rs line 79 at r3 (raw file):

Previously, matan-starkware wrote…

These go over 100 columns. Should we use subtypes type Y = XXX

This is not a type, it's a type bound. type bound aliases is an experimental feature in rust
But we plan to remove these generics once we create a test utility for sync like the one Asmaa created today for consensus


crates/papyrus_network/src/db_executor/test.rs line 171 at r3 (raw file):

Previously, matan-starkware wrote…

type TestDbExecutor = ...

This is the only place we define it, so I see no reason to create a type alias


crates/papyrus_node/src/main.rs line 143 at r3 (raw file):

Previously, matan-starkware wrote…

Why do you specify p2p?

Because there's a central sync (It doesn't have a server, but we want to connect this to the p2p sync client)

@ShahakShama ShahakShama force-pushed the shahak/register_db_executor_through_sqmr_subscriber branch from c75a7c7 to 4af5acc Compare June 20, 2024 14:34
@ShahakShama ShahakShama force-pushed the shahak/register_db_executor_through_sqmr_subscriber branch from 4af5acc to 161c1cc Compare June 24, 2024 05:53
Copy link
Contributor

@matan-starkware matan-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 6 of 6 files at r4, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @ShahakShama)


crates/papyrus_network/src/db_executor/test.rs line 171 at r3 (raw file):

Previously, ShahakShama wrote…

This is the only place we define it, so I see no reason to create a type alias

just made this function hard to read. I think it's worthwhile but won't block

@ShahakShama ShahakShama merged commit f69f005 into shahak/db_executor_channels Jun 25, 2024
16 of 17 checks passed
@ShahakShama ShahakShama deleted the shahak/register_db_executor_through_sqmr_subscriber branch June 25, 2024 04:56
ShahakShama added a commit that referenced this pull request Jun 25, 2024
* refactor(network): db executor communicates through channels

* refactor(network): fix cr comments

* refactor(network): db executor is removed from NetworkManager and runs in a separate task (#2122)

* refactor(network): db executor is removed from NetworkManager and runs in a separate task

* test(network): return network manager tests (#2124)
github-merge-queue bot pushed a commit that referenced this pull request Jun 25, 2024
* refactor(network): remove Data from db executor

* refactor(network): fix cr comments

* refactor(network): db executor communicates through channels (#2121)

* refactor(network): db executor communicates through channels

* refactor(network): fix cr comments

* refactor(network): db executor is removed from NetworkManager and runs in a separate task (#2122)

* refactor(network): db executor is removed from NetworkManager and runs in a separate task

* test(network): return network manager tests (#2124)
@github-actions github-actions bot locked and limited conversation to collaborators Jun 26, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants