-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(en): Implement gossip fetcher #371
Conversation
...and used consensus code
82f4923
to
3e0bc81
Compare
Some general notes about the PR:
|
I'd prefer to implement and test syncing from a non-genesis miniblock in a follow-up PR. AFAICT at this point, this would comprise the following items:
Most of PR should be dedicated to testing this functionality. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hey @slowli I read the PR and the code looks good, but I would def benefit from some mid-level data flow description. Maybe just a couple of sentences comparing the current approach with the one you introduce
🤖 I have created a release *beep* *boop* --- ## [18.2.0](core-v18.1.0...core-v18.2.0) (2023-11-27) ### Features * **en:** Implement gossip fetcher ([#371](#371)) ([a49b61d](a49b61d)) * **state-keeper:** reapply computational gas limit ([#544](#544)) ([698dbc3](698dbc3)) * **state-keeper:** Remove computational gas limit from boojum protocol version ([#536](#536)) ([e59a7c6](e59a7c6)) ### Bug Fixes * **core:** differentiate l2 to l1 logs tree size for pre and post boojum batches ([#538](#538)) ([1e9e556](1e9e556)) * **proof_data_handler:** Feature flag state_diff_hash check ([#545](#545)) ([0cab378](0cab378)) * **prover:** use a more performant query to get next job for FRI prover ([#527](#527)) ([2cddf3c](2cddf3c)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ## [9.1.0](prover-v9.0.0...prover-v9.1.0) (2023-12-05) ### Features * Add various metrics to the Prover subsystems ([#541](#541)) ([58a4e6c](58a4e6c)) * adds spellchecker workflow, and corrects misspelled words ([#559](#559)) ([beac0a8](beac0a8)) * **dal:** Do not load config from env in DAL crate ([#444](#444)) ([3fe1bb2](3fe1bb2)) * **en:** Implement gossip fetcher ([#371](#371)) ([a49b61d](a49b61d)) * **hyperchain:** Adding prover related commands to zk stack ([#440](#440)) ([580cada](580cada)) * **job-processor:** report attempts metrics ([#448](#448)) ([ab31f03](ab31f03)) * **merkle tree:** Allow random-order tree recovery ([#485](#485)) ([146e4cf](146e4cf)) * **witness-generator:** add logs to leaf aggregation job ([#542](#542)) ([7e95a3a](7e95a3a)) ### Bug Fixes * Change no pending batches 404 error into a success response ([#279](#279)) ([e8fd805](e8fd805)) * **ci:** Use the same nightly rust ([#530](#530)) ([67ef133](67ef133)) * **crypto:** update shivini to switch to era-cuda ([#469](#469)) ([38bb482](38bb482)) * Sync protocol version between consensus and server blocks ([#568](#568)) ([56776f9](56776f9)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: Artem Makhortov <[email protected]>
🤖 I have created a release *beep* *boop* --- ## [10.0.0](prover-v9.1.0...prover-v10.0.0) (2023-12-05) ### ⚠ BREAKING CHANGES * boojum integration ([#112](#112)) * Update to protocol version 17 ([#384](#384)) ### Features * Add various metrics to the Prover subsystems ([#541](#541)) ([58a4e6c](58a4e6c)) * adds spellchecker workflow, and corrects misspelled words ([#559](#559)) ([beac0a8](beac0a8)) * boojum integration ([#112](#112)) ([e76d346](e76d346)) * **boojum:** Adding README to prover directory ([#189](#189)) ([c175033](c175033)) * **config:** Extract everything not related to the env config from zksync_config crate ([#245](#245)) ([42c64e9](42c64e9)) * **core:** Split config definitions and deserialization ([#414](#414)) ([c7c6b32](c7c6b32)) * **dal:** Do not load config from env in DAL crate ([#444](#444)) ([3fe1bb2](3fe1bb2)) * **en:** Implement gossip fetcher ([#371](#371)) ([a49b61d](a49b61d)) * **fri-prover:** In witness - panic if protocol version is not available ([#192](#192)) ([0403749](0403749)) * **hyperchain:** Adding prover related commands to zk stack ([#440](#440)) ([580cada](580cada)) * **job-processor:** report attempts metrics ([#448](#448)) ([ab31f03](ab31f03)) * **merkle tree:** Allow random-order tree recovery ([#485](#485)) ([146e4cf](146e4cf)) * **merkle tree:** Snapshot recovery for Merkle tree ([#163](#163)) ([9e20703](9e20703)) * Rewrite server binary to use `vise` metrics ([#120](#120)) ([26ee1fb](26ee1fb)) * Update to protocol version 17 ([#384](#384)) ([ba271a5](ba271a5)) * **vm:** Move all vm versions to the one crate ([#249](#249)) ([e3fb489](e3fb489)) * **witness-generator:** add logs to leaf aggregation job ([#542](#542)) ([7e95a3a](7e95a3a)) ### Bug Fixes * Change no pending batches 404 error into a success response ([#279](#279)) ([e8fd805](e8fd805)) * **ci:** Use the same nightly rust ([#530](#530)) ([67ef133](67ef133)) * **crypto:** update deps to include circuit fixes ([#402](#402)) ([4c82015](4c82015)) * **crypto:** update shivini to switch to era-cuda ([#469](#469)) ([38bb482](38bb482)) * **crypto:** update snark-vk to be used in server and update args for proof wrapping ([#240](#240)) ([4a5c54c](4a5c54c)) * **docs:** Add links to setup-data keys ([#360](#360)) ([1d4fe69](1d4fe69)) * **path:** update gpu prover setup data path to remove extra gpu suffix ([#454](#454)) ([2e145c1](2e145c1)) * **prover-fri:** Update setup loading for node agg circuit ([#323](#323)) ([d1034b0](d1034b0)) * **prover-logging:** tasks_allowed_to_finish set to true for 1 off jobs ([#227](#227)) ([0fac66f](0fac66f)) * Sync protocol version between consensus and server blocks ([#568](#568)) ([56776f9](56776f9)) * Update prover to use the correct storage oracle ([#446](#446)) ([835dd82](835dd82)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
* feat(merkle tree): Remove enumeration index assignment from Merkle tree (#551) ## What ❔ Since enumeration indices are now fully stored in Postgres, it makes sense to not duplicate their assignment in the Merkle tree. Instead, the tree could take enum indices as inputs. ## Why ❔ This allows simplifying tree logic and unify "normal" L1 batch processing and tree recovery. (This unification is not a part of this PR; it'll be implemented separately.) ## Checklist - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. * feat(en): Support arbitrary genesis block for external nodes (#537) ## What ❔ Support non-zero genesis block specified in executor configuration. Check whether this block exists on initialization; validate its correspondence if it does, and persist consensus fields if it doesn't. ## Why ❔ This is necessary to support gossip-based syncing in practice; we likely won't back-sign all blocks in all envs. ## Checklist - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. * fix(witness_generator): Disable BWIP dependency (#573) This revert is done to facilitate boojum upgrade on mainnet2. Without this, old provers would be halt and boojum upgrade could take longer than anticipated. `waiting_for_artifacts` forced witness to wait on BWIP run. `queued` makes them run instantly. - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. - [ ] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. --------- Signed-off-by: Danil <[email protected]> Co-authored-by: Danil <[email protected]> * chore(main): release core 18.4.0 (#560) :robot: I have created a release *beep* *boop* --- ## [18.4.0](https://github.com/matter-labs/zksync-era/compare/core-v18.3.1...core-v18.4.0) (2023-12-01) ### Features * adds spellchecker workflow, and corrects misspelled words ([#559](https://github.com/matter-labs/zksync-era/issues/559)) ([beac0a8](https://github.com/matter-labs/zksync-era/commit/beac0a85bb1535b05c395057171f197cd976bf82)) * **en:** Support arbitrary genesis block for external nodes ([#537](https://github.com/matter-labs/zksync-era/issues/537)) ([15d7eaf](https://github.com/matter-labs/zksync-era/commit/15d7eaf872e222338810243865cec9dff7f6e799)) * **merkle tree:** Remove enumeration index assignment from Merkle tree ([#551](https://github.com/matter-labs/zksync-era/issues/551)) ([e2c1b20](https://github.com/matter-labs/zksync-era/commit/e2c1b20e361e6ee2f5ac69cefe75d9c5575eb2f7)) * Restore commitment test in Boojum integration ([#539](https://github.com/matter-labs/zksync-era/issues/539)) ([06f510d](https://github.com/matter-labs/zksync-era/commit/06f510d00f855ddafaebb504f7ea799700221072)) ### Bug Fixes * Change no pending batches 404 error into a success response ([#279](https://github.com/matter-labs/zksync-era/issues/279)) ([e8fd805](https://github.com/matter-labs/zksync-era/commit/e8fd805c8be7980de7676bca87cfc2d445aab9e1)) * **vm:** Expose additional types and traits ([#563](https://github.com/matter-labs/zksync-era/issues/563)) ([bd268ac](https://github.com/matter-labs/zksync-era/commit/bd268ac02bc3530c1d3247cb9496c3e13c2e52d9)) * **witness_generator:** Disable BWIP dependency ([#573](https://github.com/matter-labs/zksync-era/issues/573)) ([e05d955](https://github.com/matter-labs/zksync-era/commit/e05d955036c76a29f9b6e900872c69e20278e045)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). * ci: Runs spellcheck in merge queue. (#574) ## What ❔ Runs spellcheck in merge queue. <!-- What are the changes this PR brings about? --> <!-- Example: This PR adds a PR template to the repo. --> <!-- (For bigger PRs adding more context is appreciated) --> ## Why ❔ <!-- Why are these changes done? What goal do they contribute to? What are the principles behind them? --> <!-- Example: PR templates ensure PR reviewers, observers, and future iterators are in context about the evolution of repos. --> ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. - [ ] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. * chore: fix typo (#575) ## What ❔ - fix typo ## Why ❔ - fix typo ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. - [x] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. --------- Co-authored-by: Igor Aleksanov <[email protected]> * chore: fix typos in document (#577) ## What ❔ - fixed typo ## Why ❔ fix typos in document ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. - [x] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. --------- Co-authored-by: Igor Aleksanov <[email protected]> * chore: Fix typos (#567) Hi, I have just resolve conflict #432 Co-authored-by: Igor Aleksanov <[email protected]> * feat: Add metric to CallTracer for calculating maximum depth of the calls (#535) ## What ❔ Add metric to CallTracer for calculating maximum depth of the calls ## Why ❔ We need to know what our limits are. ## Checklist - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. * feat: Add various metrics to the Prover subsystems (#541) ## What ❔ 1. Add various metrics to the Prover subsystems, especially: * oldest block, that wasn't sent to prover(`fri_prover.oldest_unprocessed_block`) * oldest block, that didn't go through basic/leaf/node aggregation levels (`fri_prover.oldest_unprocessed_block_by_round`) * how much time is spent on waiting for available prover to send data to (`prover_fri_witness_vector_generator.prover_waiting_time) * count for attempts to send data to prover (`prover_fri_witness_vector_generator.prover_attempts_count`) 2. Refactor metrics in prover to use vise. ## Why ❔ We have some metric coverage on the prover subsystem, but it's incomplete. ## Checklist - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. * chore: fix wrong line (#592) ## What ❔ fix wrong line ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. - [x] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. * chore(docs): fixed docs typo (#588) ## What ❔ - Hello, fixed typo ## Why ❔ - fixed typo ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. - [x] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. --------- Co-authored-by: Roman Brodetski <[email protected]> * chore(docs): fix typos in document (#589) ## What ❔ Hello, I corrected the typo. ## Why ❔ - fixed typo ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. - [x] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. --------- Co-authored-by: Igor Aleksanov <[email protected]> * chore: fix typo (#587) ## What ❔ fixed typos ## Why ❔ fixed typos ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. - [x] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. --------- Co-authored-by: Igor Aleksanov <[email protected]> * chore(docs): fix broken link (#590) ## What ❔ fixed broken link ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. - [x] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. --------- Co-authored-by: Igor Aleksanov <[email protected]> * feat: faster and less noisy zk fmt (#513) ## What ❔ I've added caching to prettier and changed so that noisy output about changed files is redirected to /dev/null. `zk fmt` is 3 times faster after those changes ## Why ❔ `zk fmt` output was too verbose and we didn't use cache ## Checklist - [X] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [X] Code has been formatted via `zk fmt` and `zk lint`. * chore: the errors in the document have been correct (#583) ## What ❔ the errors in the document have been correct ## Why ❔ the errors in the document have been correct ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. - [x] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. --------- Co-authored-by: Igor Aleksanov <[email protected]> * chore(docs): the errors in the document have been corrected. (#461) ## What ❔ - the errors in the document have been corrected. ## Why ❔ - the errors in the document have been corrected. ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. --------- Co-authored-by: perekopskiy <[email protected]> * chore: update document (#601) ## What ❔ - update document ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. - [x] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. --------- Co-authored-by: Igor Aleksanov <[email protected]> Co-authored-by: perekopskiy <[email protected]> * chore: fixed typos in documentation (#603) ## What ❔ - fixed typos in documentation ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. - [x] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. --------- Co-authored-by: perekopskiy <[email protected]> * chore: remove incorrect branch prompts (#594) ## What ❔ remove incorrect branch prompts ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. - [x] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. --------- Co-authored-by: Igor Aleksanov <[email protected]> * fix: Sync protocol version between consensus and server blocks (#568) ## What ❔ Aligns the protocol version for consensus blocks with that of `SyncBlock`s. ## Why ❔ Required for gossip-based block syncing to work correctly. ## Checklist - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. - [x] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. * chore(main): release core 18.5.0 (#593) :robot: I have created a release *beep* *boop* --- ## [18.5.0](https://github.com/matter-labs/zksync-era/compare/core-v18.4.0...core-v18.5.0) (2023-12-05) ### Features * Add metric to CallTracer for calculating maximum depth of the calls ([#535](https://github.com/matter-labs/zksync-era/issues/535)) ([19c84ce](https://github.com/matter-labs/zksync-era/commit/19c84ce624d53735133fa3b12c7f980e8c14260d)) * Add various metrics to the Prover subsystems ([#541](https://github.com/matter-labs/zksync-era/issues/541)) ([58a4e6c](https://github.com/matter-labs/zksync-era/commit/58a4e6c4c22bd7f002ede1c6def0dc260706185e)) ### Bug Fixes * Sync protocol version between consensus and server blocks ([#568](https://github.com/matter-labs/zksync-era/issues/568)) ([56776f9](https://github.com/matter-labs/zksync-era/commit/56776f929f547b1a91c5b70f89e87ef7dc25c65a)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). * chore: fix link (#576) ## What ❔ fix link ## Why ❔ - fix link ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. - [x] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. --------- Co-authored-by: Igor Aleksanov <[email protected]> Co-authored-by: perekopskiy <[email protected]> * chore(main): release prover 9.1.0 (#460) :robot: I have created a release *beep* *boop* --- ## [9.1.0](https://github.com/matter-labs/zksync-era/compare/prover-v9.0.0...prover-v9.1.0) (2023-12-05) ### Features * Add various metrics to the Prover subsystems ([#541](https://github.com/matter-labs/zksync-era/issues/541)) ([58a4e6c](https://github.com/matter-labs/zksync-era/commit/58a4e6c4c22bd7f002ede1c6def0dc260706185e)) * adds spellchecker workflow, and corrects misspelled words ([#559](https://github.com/matter-labs/zksync-era/issues/559)) ([beac0a8](https://github.com/matter-labs/zksync-era/commit/beac0a85bb1535b05c395057171f197cd976bf82)) * **dal:** Do not load config from env in DAL crate ([#444](https://github.com/matter-labs/zksync-era/issues/444)) ([3fe1bb2](https://github.com/matter-labs/zksync-era/commit/3fe1bb21f8d33557353f447811ca86c60f1fe51a)) * **en:** Implement gossip fetcher ([#371](https://github.com/matter-labs/zksync-era/issues/371)) ([a49b61d](https://github.com/matter-labs/zksync-era/commit/a49b61d7769f9dd7b4cbc4905f8f8a23abfb541c)) * **hyperchain:** Adding prover related commands to zk stack ([#440](https://github.com/matter-labs/zksync-era/issues/440)) ([580cada](https://github.com/matter-labs/zksync-era/commit/580cada003bdfe2fff686a1fc3ce001b4959aa4d)) * **job-processor:** report attempts metrics ([#448](https://github.com/matter-labs/zksync-era/issues/448)) ([ab31f03](https://github.com/matter-labs/zksync-era/commit/ab31f031dfcaa7ddf296786ddccb78e8edd2d3c5)) * **merkle tree:** Allow random-order tree recovery ([#485](https://github.com/matter-labs/zksync-era/issues/485)) ([146e4cf](https://github.com/matter-labs/zksync-era/commit/146e4cf2f8d890ff0a8d33229e224442e14be437)) * **witness-generator:** add logs to leaf aggregation job ([#542](https://github.com/matter-labs/zksync-era/issues/542)) ([7e95a3a](https://github.com/matter-labs/zksync-era/commit/7e95a3a66ea48be7b6059d34630e22c503399bdf)) ### Bug Fixes * Change no pending batches 404 error into a success response ([#279](https://github.com/matter-labs/zksync-era/issues/279)) ([e8fd805](https://github.com/matter-labs/zksync-era/commit/e8fd805c8be7980de7676bca87cfc2d445aab9e1)) * **ci:** Use the same nightly rust ([#530](https://github.com/matter-labs/zksync-era/issues/530)) ([67ef133](https://github.com/matter-labs/zksync-era/commit/67ef1339d42786efbeb83c22fac99f3bf5dd4380)) * **crypto:** update shivini to switch to era-cuda ([#469](https://github.com/matter-labs/zksync-era/issues/469)) ([38bb482](https://github.com/matter-labs/zksync-era/commit/38bb4823c7b5e0e651d9f531feede66c24afd19f)) * Sync protocol version between consensus and server blocks ([#568](https://github.com/matter-labs/zksync-era/issues/568)) ([56776f9](https://github.com/matter-labs/zksync-era/commit/56776f929f547b1a91c5b70f89e87ef7dc25c65a)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: Artem Makhortov <[email protected]> * chore(main): release prover 10.0.0 (#608) :robot: I have created a release *beep* *boop* --- ## [10.0.0](https://github.com/matter-labs/zksync-era/compare/prover-v9.1.0...prover-v10.0.0) (2023-12-05) ### ⚠ BREAKING CHANGES * boojum integration ([#112](https://github.com/matter-labs/zksync-era/issues/112)) * Update to protocol version 17 ([#384](https://github.com/matter-labs/zksync-era/issues/384)) ### Features * Add various metrics to the Prover subsystems ([#541](https://github.com/matter-labs/zksync-era/issues/541)) ([58a4e6c](https://github.com/matter-labs/zksync-era/commit/58a4e6c4c22bd7f002ede1c6def0dc260706185e)) * adds spellchecker workflow, and corrects misspelled words ([#559](https://github.com/matter-labs/zksync-era/issues/559)) ([beac0a8](https://github.com/matter-labs/zksync-era/commit/beac0a85bb1535b05c395057171f197cd976bf82)) * boojum integration ([#112](https://github.com/matter-labs/zksync-era/issues/112)) ([e76d346](https://github.com/matter-labs/zksync-era/commit/e76d346d02ded771dea380aa8240da32119d7198)) * **boojum:** Adding README to prover directory ([#189](https://github.com/matter-labs/zksync-era/issues/189)) ([c175033](https://github.com/matter-labs/zksync-era/commit/c175033b48a8da4969d88b6850dd0247c4004794)) * **config:** Extract everything not related to the env config from zksync_config crate ([#245](https://github.com/matter-labs/zksync-era/issues/245)) ([42c64e9](https://github.com/matter-labs/zksync-era/commit/42c64e91e13b6b37619f1459f927fa046ef01097)) * **core:** Split config definitions and deserialization ([#414](https://github.com/matter-labs/zksync-era/issues/414)) ([c7c6b32](https://github.com/matter-labs/zksync-era/commit/c7c6b321a63dbcc7f1af045aa7416e697beab08f)) * **dal:** Do not load config from env in DAL crate ([#444](https://github.com/matter-labs/zksync-era/issues/444)) ([3fe1bb2](https://github.com/matter-labs/zksync-era/commit/3fe1bb21f8d33557353f447811ca86c60f1fe51a)) * **en:** Implement gossip fetcher ([#371](https://github.com/matter-labs/zksync-era/issues/371)) ([a49b61d](https://github.com/matter-labs/zksync-era/commit/a49b61d7769f9dd7b4cbc4905f8f8a23abfb541c)) * **fri-prover:** In witness - panic if protocol version is not available ([#192](https://github.com/matter-labs/zksync-era/issues/192)) ([0403749](https://github.com/matter-labs/zksync-era/commit/040374900656c854a7b9de32e5dbaf47c1c47889)) * **hyperchain:** Adding prover related commands to zk stack ([#440](https://github.com/matter-labs/zksync-era/issues/440)) ([580cada](https://github.com/matter-labs/zksync-era/commit/580cada003bdfe2fff686a1fc3ce001b4959aa4d)) * **job-processor:** report attempts metrics ([#448](https://github.com/matter-labs/zksync-era/issues/448)) ([ab31f03](https://github.com/matter-labs/zksync-era/commit/ab31f031dfcaa7ddf296786ddccb78e8edd2d3c5)) * **merkle tree:** Allow random-order tree recovery ([#485](https://github.com/matter-labs/zksync-era/issues/485)) ([146e4cf](https://github.com/matter-labs/zksync-era/commit/146e4cf2f8d890ff0a8d33229e224442e14be437)) * **merkle tree:** Snapshot recovery for Merkle tree ([#163](https://github.com/matter-labs/zksync-era/issues/163)) ([9e20703](https://github.com/matter-labs/zksync-era/commit/9e2070380e6720d84563a14a2246fc18fdb1f8f9)) * Rewrite server binary to use `vise` metrics ([#120](https://github.com/matter-labs/zksync-era/issues/120)) ([26ee1fb](https://github.com/matter-labs/zksync-era/commit/26ee1fbb16cbd7c4fad334cbc6804e7d779029b6)) * Update to protocol version 17 ([#384](https://github.com/matter-labs/zksync-era/issues/384)) ([ba271a5](https://github.com/matter-labs/zksync-era/commit/ba271a5f34d64d04c0135b8811685b80f26a8c32)) * **vm:** Move all vm versions to the one crate ([#249](https://github.com/matter-labs/zksync-era/issues/249)) ([e3fb489](https://github.com/matter-labs/zksync-era/commit/e3fb4894d08aa98a84e64eaa95b51001055cf911)) * **witness-generator:** add logs to leaf aggregation job ([#542](https://github.com/matter-labs/zksync-era/issues/542)) ([7e95a3a](https://github.com/matter-labs/zksync-era/commit/7e95a3a66ea48be7b6059d34630e22c503399bdf)) ### Bug Fixes * Change no pending batches 404 error into a success response ([#279](https://github.com/matter-labs/zksync-era/issues/279)) ([e8fd805](https://github.com/matter-labs/zksync-era/commit/e8fd805c8be7980de7676bca87cfc2d445aab9e1)) * **ci:** Use the same nightly rust ([#530](https://github.com/matter-labs/zksync-era/issues/530)) ([67ef133](https://github.com/matter-labs/zksync-era/commit/67ef1339d42786efbeb83c22fac99f3bf5dd4380)) * **crypto:** update deps to include circuit fixes ([#402](https://github.com/matter-labs/zksync-era/issues/402)) ([4c82015](https://github.com/matter-labs/zksync-era/commit/4c820150714dfb01c304c43e27f217f17deba449)) * **crypto:** update shivini to switch to era-cuda ([#469](https://github.com/matter-labs/zksync-era/issues/469)) ([38bb482](https://github.com/matter-labs/zksync-era/commit/38bb4823c7b5e0e651d9f531feede66c24afd19f)) * **crypto:** update snark-vk to be used in server and update args for proof wrapping ([#240](https://github.com/matter-labs/zksync-era/issues/240)) ([4a5c54c](https://github.com/matter-labs/zksync-era/commit/4a5c54c48bbc100c29fa719c4b1dc3535743003d)) * **docs:** Add links to setup-data keys ([#360](https://github.com/matter-labs/zksync-era/issues/360)) ([1d4fe69](https://github.com/matter-labs/zksync-era/commit/1d4fe697e4e98a8e64642cde4fe202338ce5ec61)) * **path:** update gpu prover setup data path to remove extra gpu suffix ([#454](https://github.com/matter-labs/zksync-era/issues/454)) ([2e145c1](https://github.com/matter-labs/zksync-era/commit/2e145c192b348b2756acf61fac5bfe0ca5a6575f)) * **prover-fri:** Update setup loading for node agg circuit ([#323](https://github.com/matter-labs/zksync-era/issues/323)) ([d1034b0](https://github.com/matter-labs/zksync-era/commit/d1034b05754219b603508ef79c114d908c94c1e9)) * **prover-logging:** tasks_allowed_to_finish set to true for 1 off jobs ([#227](https://github.com/matter-labs/zksync-era/issues/227)) ([0fac66f](https://github.com/matter-labs/zksync-era/commit/0fac66f5ff86cc801ea0bb6f9272cb397cd03a95)) * Sync protocol version between consensus and server blocks ([#568](https://github.com/matter-labs/zksync-era/issues/568)) ([56776f9](https://github.com/matter-labs/zksync-era/commit/56776f929f547b1a91c5b70f89e87ef7dc25c65a)) * Update prover to use the correct storage oracle ([#446](https://github.com/matter-labs/zksync-era/issues/446)) ([835dd82](https://github.com/matter-labs/zksync-era/commit/835dd828ef5610a446ec8c456e4df1def0e213ab)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). * refactor: Removed protobuf encoding from zksync_types (#562) ## What ❔ Removed protobuf encoding from zksync_types. ## Why ❔ To make zksync_types have less dependencies. * fix: use powers array in plonkSetup function (#508) ## What ❔ This PR modifies the `plonkSetup` function in `run.ts` to use the `powers` array when downloading key files. ## Why ❔ Previously, the function forget to use the argument values `powers`. Now, it will download keys for any powers specified in the `powers` array. ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [NA] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. --------- Co-authored-by: Igor Aleksanov <[email protected]> * fix: Fix database connections in house keeper (#610) ## What ❔ Use correct connections for databases in house keeper. ## Why ❔ Databases are divided in 2 on mainnet and testnet ## Checklist - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. - [x] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. * feat(contract-verifier): Support verification for zksolc v1.3.17 (#606) ## What ❔ Adds support for zksolc v1.3.17 to contract-verifier. ## Why ❔ Contract-verifier should support latest version ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. - [x] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. * chore(main): release core 18.6.0 (#613) :robot: I have created a release *beep* *boop* --- ## [18.6.0](https://github.com/matter-labs/zksync-era/compare/core-v18.5.0...core-v18.6.0) (2023-12-05) ### Features * **contract-verifier:** Support verification for zksolc v1.3.17 ([#606](https://github.com/matter-labs/zksync-era/issues/606)) ([b65fedd](https://github.com/matter-labs/zksync-era/commit/b65fedd6894497a4c9fbf38d558ccfaca535d1d2)) ### Bug Fixes * Fix database connections in house keeper ([#610](https://github.com/matter-labs/zksync-era/issues/610)) ([aeaaecb](https://github.com/matter-labs/zksync-era/commit/aeaaecb54b6bd3f173727531418dc242357b2aee)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). * chore: Mainnet upgrade calldata (#564) ## What ❔ Includes mainnet upgrade preparation as well as some minor fixes for the upgrade tool ## Why ❔ <!-- Why are these changes done? What goal do they contribute to? What are the principles behind them? --> <!-- Example: PR templates ensure PR reviewers, observers, and future iterators are in context about the evolution of repos. --> ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [ ] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [ ] Code has been formatted via `zk fmt` and `zk lint`. --------- Co-authored-by: koloz <[email protected]> * chore: Update generated Prover FRI CPU setup-data keys from branch main (#609) "Update generated Prover FRI CPU setup-data keys from branch main" * perf(external-node): Use async miniblock sealing in external IO (#611) ## What ❔ External IO uses async miniblock sealing. ## Why ❔ Execution of transactions and miniblock sealing (writing data to postgres) happen in parallel so the perfomance is better. ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. - [x] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. * chore: fix document path (#615) ## What ❔ fix document path ## Why ❔ <!-- Why are these changes done? What goal do they contribute to? What are the principles behind them? --> <!-- Example: PR templates ensure PR reviewers, observers, and future iterators are in context about the evolution of repos. --> ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. - [x] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. --------- Co-authored-by: Igor Aleksanov <[email protected]> * chore: Remove era-reviewers from codeowners (#618) ## What ❔ Removes era-reviewers group from codeowners. ## Why ❔ - Too noisy. - We have internal processes for that anyways. ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [ ] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [ ] Code has been formatted via `zk fmt` and `zk lint`. - [ ] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. * chore(main): release core 18.6.1 (#616) :robot: I have created a release *beep* *boop* --- ## [18.6.1](https://github.com/matter-labs/zksync-era/compare/core-v18.6.0...core-v18.6.1) (2023-12-06) ### Performance Improvements * **external-node:** Use async miniblock sealing in external IO ([#611](https://github.com/matter-labs/zksync-era/issues/611)) ([5cf7210](https://github.com/matter-labs/zksync-era/commit/5cf7210dc77bb615944352f23ed39fad324b914f)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). * feat(hyperchain-wizard): zkStack CLI GPU support (#612) ## What ❔ Support for creating zk hyperchain via zk cli with GPU-based provers ## Why ❔ ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [X] Code has been formatted via `zk fmt` and `zk lint`. - [ ] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. --------- Co-authored-by: Igor Borodin <[email protected]> * fix: Cursor not moving correctly after poll in `get_filter_changes` (#546) ## What ❔ When polling filter changes, add 1 to actual from_block value ## Why ❔ Otherwise, last block that was included in poll will be included to the next one. ## Checklist - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. --------- Co-authored-by: Fedor Sakharov <[email protected]> * fix: update google cloud dependencies that do not depend on rsa (#622) ## What ❔ This PR updates the dependencies of `google-cloud-storage` and `google-cloud-auth`. The changes are as follows: - From google-cloud-storage = "0.12.0" to google-cloud-storage = "0.15.0" - From google-cloud-auth = "0.11.0" to google-cloud-auth = "0.13.0" Relevant google-cloud changes: https://github.com/yoshidan/google-cloud-rust/pull/217 ## Why ❔ <!-- Why are these changes done? What goal do they contribute to? What are the principles behind them? --> <!-- Example: PR templates ensure PR reviewers, observers, and future iterators are in context about the evolution of repos. --> The primary reason for these updates is to address a security vulnerability associated with the `rsa` crate, as indicated by a recent `cargo-deny` check. The vulnerability (Marvin Attack, RUSTSEC-2023-0071) was detected in rsa v0.6.1, which is a dependency of `google-cloud-storage v0.12.0`. By updating to `google-cloud-storage v0.15.0`, we eliminate the use of the `rsa` crate, as the newer version of `google-cloud-storage` does not depend on it. Similarly, `google-cloud-auth` is updated for compatibility. Cargo deny output: ``` error[vulnerability]: Marvin Attack: potential key recovery through timing sidechannels ┌─ /Users/dustinbrickwood/Documents/dev/dut/forks/foundry-zksync/Cargo.lock:759:1 │ 759 │ rsa 0.6.1 registry+https://github.com/rust-lang/crates.io-index │ --------------------------------------------------------------- security vulnerability detected │ = ID: RUSTSEC-2023-0071 = Advisory: https://rustsec.org/advisories/RUSTSEC-2023-0071 = ### Impact Due to a non-constant-time implementation, information about the private key is leaked through timing information which is observable over the network. An attacker may be able to use that information to recover the key. ### Patches No patch is yet available, however work is underway to migrate to a fully constant-time implementation. ### Workarounds The only currently available workaround is to avoid using the `rsa` crate in settings where attackers are able to observe timing information, e.g. local use on a non-compromised computer is fine. ### References This vulnerability was discovered as part of the "[Marvin Attack]", which revealed several implementations of RSA including OpenSSL had not properly mitigated timing sidechannel attacks. [Marvin Attack]: https://people.redhat.com/~hkario/marvin/ = Announcement: https://github.com/RustCrypto/RSA/issues/19#issuecomment-1822995643 = Solution: No safe upgrade is available! = rsa v0.6.1 └── google-cloud-storage v0.12.0 └── zksync_object_store v0.1.0 ├── zksync_core v0.1.0 │ └── era_test_node v0.1.0-alpha.12 │ └── era_revm v0.0.1-alpha │ ├── foundry-common v0.2.0 │ │ ├── anvil v0.2.0 │ │ │ ├── (dev) forge v0.2.0 │ │ │ └── (dev) zkforge v0.2.0 │ │ ├── cast v0.2.0 │ │ ├── chisel v0.2.0 │ │ ├── forge v0.2.0 (*) │ │ ├── foundry-cli v0.2.0 │ │ │ ├── cast v0.2.0 (*) │ │ │ ├── chisel v0.2.0 (*) │ │ │ ├── forge v0.2.0 (*) │ │ │ ├── zkcast v0.2.0 │ │ │ │ └── zkforge v0.2.0 (*) │ │ │ └── zkforge v0.2.0 (*) │ │ ├── foundry-debugger v0.2.0 │ │ │ ├── forge v0.2.0 (*) │ │ │ ├── foundry-cli v0.2.0 (*) │ │ │ └── zkforge v0.2.0 (*) │ │ ├── foundry-evm v0.2.0 │ │ │ ├── anvil v0.2.0 (*) │ │ │ ├── anvil-core v0.2.0 │ │ │ │ └── anvil v0.2.0 (*) │ │ │ ├── cast v0.2.0 (*) │ │ │ ├── chisel v0.2.0 (*) │ │ │ ├── forge v0.2.0 (*) │ │ │ ├── foundry-cli v0.2.0 (*) │ │ │ ├── foundry-debugger v0.2.0 (*) │ │ │ ├── zkcast v0.2.0 (*) │ │ │ └── zkforge v0.2.0 (*) │ │ ├── foundry-test-utils v0.2.0 │ │ │ ├── (dev) cast v0.2.0 (*) │ │ │ ├── (dev) forge v0.2.0 (*) │ │ │ ├── (dev) zkcast v0.2.0 (*) │ │ │ └── (dev) zkforge v0.2.0 (*) │ │ ├── (dev) foundry-utils v0.2.0 │ │ │ ├── anvil v0.2.0 (*) │ │ │ ├── anvil-core v0.2.0 (*) │ │ │ ├── cast v0.2.0 (*) │ │ │ ├── chisel v0.2.0 (*) │ │ │ ├── forge v0.2.0 (*) │ │ │ ├── forge-doc v0.2.0 │ │ │ │ ├── forge v0.2.0 (*) │ │ │ │ └── zkforge v0.2.0 (*) │ │ │ ├── foundry-cli v0.2.0 (*) │ │ │ ├── foundry-debugger v0.2.0 (*) │ │ │ ├── (dev) foundry-evm v0.2.0 (*) │ │ │ ├── foundry-test-utils v0.2.0 (*) │ │ │ ├── zkcast v0.2.0 (*) │ │ │ └── zkforge v0.2.0 (*) │ │ ├── zkcast v0.2.0 (*) │ │ └── zkforge v0.2.0 (*) │ └── foundry-evm v0.2.0 (*) └── zksync_prover_utils v0.1.0 ├── zksync_core v0.1.0 (*) └── zksync_verification_key_generator_and_server v0.1.0 └── zksync_core v0.1.0 (*) ``` ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. - [x] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. * docs: Include command to create rich L2 wallets. (#569) ## What ❔ Improve documentation by including the command to create rich L2 wallets. ## Why ❔ Save other people time figuring out the exact invocation. ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [ ] Code has been formatted via `zk fmt` and `zk lint`. - [ ] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. * chore: Enforce uniform import structure (#617) ## What ❔ ...using `zk fmt` command by suppling relevant command-line args to rustfmt. These args work on stable Rust (at least for now) despite being unstable. ## Why ❔ More structured imports are easier to read. ## Checklist - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Code has been formatted via `zk fmt` and `zk lint`. - [x] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. * fix(job-processor): `max_attepts_reached` metric (#626) ## What ❔ `max_attepts_reached` metric is now reported on job start rather failure. With this change metric will be reported not only if last attempt failed but also if it got stuck/stopped/etc. ## Why ❔ Reporting `max_attepts_reached` metric for all cases. ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. - [x] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. * chore(vm): Expose more pubs and make inmemory clonable (#632) ## What ❔ Expose AppFramestack public and make InMemoryStorage Clonable. ## Why ❔ For supporting era-test-node ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [ ] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [ ] Code has been formatted via `zk fmt` and `zk lint`. - [ ] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. --------- Signed-off-by: Danil <[email protected]> * chore: remove old witness generator (#619) * fix: improve docs repositories (#570) ## What ❔ * Add new/missing zkSync repositories * Add missing descriptions * Remove deprecated repositories ## Why ❔ To make the list up-to-date ## Checklist - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. - [x] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. * docs(setup): Added TL;DR instructions for new zkstack setup (#621) ## What ❔ * Added a TL;DR set of instructions needed to setup the system to run the zk stack from scratch. ## Why ❔ * To have a list of commands in one place. * chore: upgrades local test network to cancun+deneb compatible one (#580) ## What ❔ Upgrades local testnet to `Cancun+Deneb` compatible one. So far: Cancun gets enabled: ``` 2023-12-01 21:57:49 INFO [12-01|20:57:49.152] Merge configured: 2023-12-01 21:57:49 INFO [12-01|20:57:49.152] - Hard-fork specification: https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/paris.md 2023-12-01 21:57:49 INFO [12-01|20:57:49.152] - Network known to be merged: true 2023-12-01 21:57:49 INFO [12-01|20:57:49.152] - Total terminal difficulty: 0 2023-12-01 21:57:49 INFO [12-01|20:57:49.152] 2023-12-01 21:57:49 INFO [12-01|20:57:49.152] Post-Merge hard forks (timestamp based): 2023-12-01 21:57:49 INFO [12-01|20:57:49.152] - Shanghai: @1701464272 (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/shanghai.md) 2023-12-01 21:57:49 INFO [12-01|20:57:49.152] - Cancun: @1701464272 ``` New network has been built into CI workflows. ## Why ❔ <!-- Why are these changes done? What goal do they contribute to? What are the principles behind them? --> <!-- Example: PR templates ensure PR reviewers, observers, and future iterators are in context about the evolution of repos. --> ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [ ] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [ ] Code has been formatted via `zk fmt` and `zk lint`. - [ ] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. * chore(zk init): Removed plonk setup (#638) ## What ❔ * Removed plonk setup (where we downloaded the CRS keys) ## Why ❔ * It is needed only for prover setup (and then this is handled within the prover_setup.ts directly) * it will save a lot of network bandwidth and time during zk stack setup * chore(CI): Speeding up docker builds in CI (#640) ## What ❔ * Removed zk contract compilation that was run before prover docker builds * Removed docker builds for old prover and old circuit_synthesizer * Downloading CRS file only for the snark wrapper / compressor job ## Why ❔ * to speed up docker CI that runs on every PR * chore(zk): finishing migration to docker compose (#646) ## What ❔ * replaced docker-compose with docker compose ## Why ❔ * we moved to use docker compose (instead of docker-compose) - which is a newer version * chore(ci): Pre-download compilers, as a workaround for old hardhat plugins (#645) ## What ❔ Add hack way to pre-download needed compileres ## Why ❔ Hardhat plugins, which we are using right now, are pretty often rate limited by github for downloading compilers, due to unoptimized download procedure. This issue was fixed in later versions of plugins, but we can't update _some_ of the hardhat plugins due to changes in deps for them. This pre-download will help us till the time we can update them. ## Checklist - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [ ] Code has been formatted via `zk fmt` and `zk lint`. - [ ] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. * feat(en): Remove `SyncBlock.root_hash` (#633) ## What ❔ Removes `root_hash` field from `SyncBlock`. ## Why ❔ It's not used anywhere, set to a dummy value (miniblock hash) and doesn't make sense in general (state hashes are computed on L1 batch level, not miniblock level). ## Checklist - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. - [x] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. * docs: New protocol specification (#641) ## What ❔ New docs, focusing on protocol specs. Note: don't merge yet, waiting for review. ## Why ❔ We want to have good docs so people can understand the zkVM and its benefits. ## Checklist - [ X] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ X] Tests for the changes have been added / updated. - [ X] Documentation comments have been added / updated. - [ X] Code has been formatted via `zk fmt` and `zk lint`. - [ ] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. --------- Co-authored-by: kelemeno <[email protected]> Co-authored-by: MexicanAce <[email protected]> Co-authored-by: Jack Hamer <[email protected]> Co-authored-by: Marcin M <[email protected]> Co-authored-by: Fedor Sakharov <[email protected]> * feat(zk tool): Added yarn & directory checks (#188) # What ❔ * zk will now print a warning if you run it from outside of $ZKSYNC_HOME directory * it will also warn you if your yarn has a wrong version ## Why ❔ * we use ZKSYNC_HOME to figure out which binaries to use, so if you are working on couple versions of era in multiple different directories, and forget to update ZKSYNC_HOME, then you might be pushing wrong binaries without knowing. * if your version of yarn is incorrect, you're going to get a lot of strange typescript compilation errors along the way. These are only printed as warnings, they don't prevent you from running the tool. ![image](https://github.com/matter-labs/zksync-era/assets/128217157/e8186e36-953d-4307-9449-048bdd589321) * fix: follow up metrics fixes (#648) ## What ❔ Fixes for previous metrics PRs. * Fix oldest unproved block by round (we have `manually_skipped` status in DB, is this correct?), SQL query and number of rounds * Rename some API server metrics * May filter count work incorrect? ## Why ❔ For metrics to work correctly. ## Checklist - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. - [x] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. * chore(ci): Pre-download one more version of viper compiler (#652) ## What ❔ Pre-download one more version of viper compiler ## Why ❔ Workaround for rate limiting ## Checklist - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [ ] Code has been formatted via `zk fmt` and `zk lint`. - [ ] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. * feat(en): Check block hash correspondence (#572) ## What ❔ Checks that the block hash received by EN corresponds to the locally computed block hash. The check occurs reasonably early (i.e., before the block is executed). ## Why ❔ Allows to ensure that there's nothing fundamentally wrong with EN implementation. ## Checklist - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. - [x] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. * fix: fix docs error (#635) ## What ❔ fix docs error ## Why ❔ <!-- Why are these changes done? What goal do they contribute to? What are the principles behind them? --> <!-- Example: PR templates ensure PR reviewers, observers, and future iterators are in context about the evolution of repos. --> ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. - [x] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. --------- Co-authored-by: Igor Aleksanov <[email protected]> Co-authored-by: Igor Borodin <[email protected]> Co-authored-by: AnastasiiaVashchuk <[email protected]> * chore(core/lib): typo fix (#624) few minor typo fix. 3 comments, 1 error message --------- Co-authored-by: AnastasiiaVashchuk <[email protected]> * chore: remove tslint related comments (#636) ## What ❔ This pull request aims to remove TSLint-related comments and configurations from the codebase. ## Why ❔ The removal is necessary because the repository has transitioned to ESLint, TSLint-related comments and configurations obsolete. This cleanup ensures a more streamlined and consistent codebase, aligning with the migration to ESLint for improved code quality and maintainability. ## Checklist - [X] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [X] Tests for the changes have been added / updated. - [X] Documentation comments have been added / updated. - [X] Code has been formatted via `zk fmt` and `zk lint`. - [X] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. --------- Co-authored-by: AnastasiiaVashchuk <[email protected]> * feat: Snapshot Creator (#498) ## What ❔ Snapshot creator is small command line tool for creating a snapshot of zkSync node for EN node to be able to initialize to a certain L1 Batch. Snapshots do not contain full transactions history, but rather a minimal subset of information needed to bootstrap EN node. ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [X] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [X] Tests for the changes have been added / updated. - [X] Documentation comments have been added / updated. - [X] Code has been formatted via `zk fmt` and `zk lint`. * chore(ci): Pre-download more versions of compilers + less verbose (#653) ## What ❔ Pre-download more versions of compilers ## Checklist - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [ ] Code has been formatted via `zk fmt` and `zk lint`. - [ ] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. * feat(contract-verifier): Add zksolc v1.3.18 (#654) ## What ❔ Adds zksolc v1.3.18 to contract verifier ## Why ❔ Adds zksolc v1.3.18 to contract verifier ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [ ] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [ ] Code has been formatted via `zk fmt` and `zk lint`. - [ ] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. * fix: Follow up metrics fixes vol.2 (#656) ## What ❔ Fix oldest not generated batch query ## Why ❔ For metrics to work correctly ## Checklist - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. - [x] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. * fix: removing sqlx check from pre-push (#658) Removing this check for now, I will re-add is as an opt-in check, probably with other checks bundled in. It's best not to require built project on every push. * chore(main): release core 18.7.0 (#620) :robot: I have created a release *beep* *boop* --- ## [18.7.0](https://github.com/matter-labs/zksync-era/compare/core-v18.6.1...core-v18.7.0) (2023-12-12) ### Features * **contract-verifier:** Add zksolc v1.3.18 ([#654](https://github.com/matter-labs/zksync-era/issues/654)) ([77f91fe](https://github.com/matter-labs/zksync-era/commit/77f91fe253a0876e56de4aee47071fe249386fc7)) * **en:** Check block hash correspondence ([#572](https://github.com/matter-labs/zksync-era/issues/572)) ([28f5642](https://github.com/matter-labs/zksync-era/commit/28f5642c35800997879bc549fca9e960c4516d21)) * **en:** Remove `SyncBlock.root_hash` ([#633](https://github.com/matter-labs/zksync-era/issues/633)) ([d4cc6e5](https://github.com/matter-labs/zksync-era/commit/d4cc6e564642b4c49ef4a546cd1c86821327683c)) * Snapshot Creator ([#498](https://github.com/matter-labs/zksync-era/issues/498)) ([270edee](https://github.com/matter-labs/zksync-era/commit/270edee34402ecbd1761bc1fca559ef2205f71e8)) ### Bug Fixes * Cursor not moving correctly after poll in `get_filter_changes` ([#546](https://github.com/matter-labs/zksync-era/issues/546)) ([ec5907b](https://github.com/matter-labs/zksync-era/commit/ec5907b70ff7d868a05b685a1641d96dc4fa9d69)) * fix docs error ([#635](https://github.com/matter-labs/zksync-era/issues/635)) ([883c128](https://github.com/matter-labs/zksync-era/commit/883c1282f7771fb16a41d45391b74243021271e3)) * follow up metrics fixes ([#648](https://github.com/matter-labs/zksync-era/issues/648)) ([a317c7a](https://github.com/matter-labs/zksync-era/commit/a317c7ab68219cb376d08c8d1ec210c63b3c269f)) * Follow up metrics fixes vol.2 ([#656](https://github.com/matter-labs/zksync-era/issues/656)) ([5c1aea2](https://github.com/matter-labs/zksync-era/commit/5c1aea2a94d7eded26c3a4ae4973ff983c15e7fa)) * **job-processor:** `max_attepts_reached` metric ([#626](https://github.com/matter-labs/zksync-era/issues/626)) ([dd9b308](https://github.com/matter-labs/zksync-era/commit/dd9b308be9b0a6e37aad75f6f54b98e30a2ae14e)) * update google cloud dependencies that do not depend on rsa ([#622](https://github.com/matter-labs/zksync-era/issues/622)) ([8a8cad6](https://github.com/matter-labs/zksync-era/commit/8a8cad6ce62f2d34bb34adcd956f6920c08f94b8)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). * ci: removes docker-compose-runner.yml (#649) ## What ❔ <!-- What are the changes this PR brings about? --> <!-- Example: This PR adds a PR template to the repo. --> <!-- (For bigger PRs adding more context is appreciated) --> ## Why ❔ <!-- Why are these changes done? What goal do they contribute to? What are the principles behind them? --> <!-- Example: PR templates ensure PR reviewers, observers, and future iterators are in context about the evolution of repos. --> ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [ ] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [ ] Code has been formatted via `zk fmt` and `zk lint`. - [ ] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. * chore(en): adds version metric collecting (#655) ## What ❔ This PR adds a metric that logs release version and protocol version of the external node. That is needed for provisioning purposes. ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [ ] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [ ] Code has been formatted via `zk fmt` and `zk lint`. - [ ] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. * ci(local-node): replace deprecated node setup (#642) ## What ❔ As in commit 33174aa59 and `docker/zk-environment/Dockerfile` use the repo to setup node, instead of the `curl | sh`. <!-- What are the changes this PR brings about? --> <!-- Example: This PR adds a PR template to the repo. --> <!-- (For bigger PRs adding more context is appreciated) --> ## Why ❔ The old method is deprecated and it saves one minute waiting time instead of the deprecation banner being displayed. <!-- Why are these changes done? What goal do they contribute to? What are the principles behind them? --> <!-- Example: PR templates ensure PR reviewers, observers, and future iterators are in context about the evolution of repos. --> ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [ ] Code has been formatted via `zk fmt` and `zk lint`. - [ ] Spellcheck has been run via `cargo spellcheck --cfg=./spellcheck/era.cfg --code 1`. Signed-off-by: Harald Hoyer <[email protected]> * chore(en): Add sepolia en config (#663) ## What ❔ - marks goerli testnet EN config as deprecated - adds sepolia testnet EN config - removes `EN_BOOTLOADER_HASH`, `EN_DEFAULT_AA_HASH` from configs as these vars are not used anymore ## Why ❔ Prepare docs for sepolia testnet EN launch ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Documentation comme…
What ❔
...i.e., a fetcher component that would use gossip network instead of JSON-RPC API.
Fixes BFT-326 and BFT-368.
Why ❔
This can be used by external nodes to sync with the main node.
Checklist
zk fmt
andzk lint
.