Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(en): Implement gossip fetcher #371

Merged
merged 43 commits into from
Nov 23, 2023
Merged

Conversation

slowli
Copy link
Contributor

@slowli slowli commented Nov 1, 2023

What ❔

...i.e., a fetcher component that would use gossip network instead of JSON-RPC API.
Fixes BFT-326 and BFT-368.

Why ❔

This can be used by external nodes to sync with the main node.

Checklist

  • PR title corresponds to the body of PR (we generate changelog entries from PRs).
  • Tests for the changes have been added / updated.
  • Documentation comments have been added / updated.
  • Code has been formatted via zk fmt and zk lint.

@slowli slowli force-pushed the aov-bft-368-implement-gossip-fetcher branch from 82f4923 to 3e0bc81 Compare November 2, 2023 13:07
@slowli
Copy link
Contributor Author

slowli commented Nov 3, 2023

Some general notes about the PR:

  • The new fetcher is not actually integrated into EN in order to not make the PR any larger. Correspondingly, I'm not 100% sure it'll actually work end-to-end. (This PR does include tests that spin up a gossip network and pass blocks using it – not really end-to-end testing, but maybe somewhat reassuring.)
  • We'd want to implement syncing from a certain L2 block number, rather than from genesis. This will probably require changes both on server and consensus sides. Again, I've decided not to include this logic to not bloat this PR.
  • Ideally, I'd like to separate adding new data for miniblocks to a separate PR. However, this doesn't seem to be possible (at least without major crutches) given that we want to have quorum certificate validation.

@slowli
Copy link
Contributor Author

slowli commented Nov 17, 2023

I'd prefer to implement and test syncing from a non-genesis miniblock in a follow-up PR. AFAICT at this point, this would comprise the following items:

  • Load the first block with consensus fields from Postgres when a node starts and comparing it against the reference block. If there are no blocks with consensus fields, the node will persist it from the reference.
  • Use this first block in BlockStore::first_block() rather than block 0.

Most of PR should be dedicated to testing this functionality.

pompon0
pompon0 previously approved these changes Nov 17, 2023
RomanBrodetski
RomanBrodetski previously approved these changes Nov 22, 2023
Copy link
Collaborator

@RomanBrodetski RomanBrodetski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hey @slowli I read the PR and the code looks good, but I would def benefit from some mid-level data flow description. Maybe just a couple of sentences comparing the current approach with the one you introduce

@slowli slowli added this pull request to the merge queue Nov 23, 2023
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Nov 23, 2023
@slowli slowli added this pull request to the merge queue Nov 23, 2023
Merged via the queue into main with commit a49b61d Nov 23, 2023
31 checks passed
@slowli slowli deleted the aov-bft-368-implement-gossip-fetcher branch November 23, 2023 09:42
github-merge-queue bot pushed a commit that referenced this pull request Nov 27, 2023
🤖 I have created a release *beep* *boop*
---


##
[18.2.0](core-v18.1.0...core-v18.2.0)
(2023-11-27)


### Features

* **en:** Implement gossip fetcher
([#371](#371))
([a49b61d](a49b61d))
* **state-keeper:** reapply computational gas limit
([#544](#544))
([698dbc3](698dbc3))
* **state-keeper:** Remove computational gas limit from boojum protocol
version ([#536](#536))
([e59a7c6](e59a7c6))


### Bug Fixes

* **core:** differentiate l2 to l1 logs tree size for pre and post
boojum batches
([#538](#538))
([1e9e556](1e9e556))
* **proof_data_handler:** Feature flag state_diff_hash check
([#545](#545))
([0cab378](0cab378))
* **prover:** use a more performant query to get next job for FRI prover
([#527](#527))
([2cddf3c](2cddf3c))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
github-merge-queue bot pushed a commit that referenced this pull request Dec 5, 2023
🤖 I have created a release *beep* *boop*
---


##
[9.1.0](prover-v9.0.0...prover-v9.1.0)
(2023-12-05)


### Features

* Add various metrics to the Prover subsystems
([#541](#541))
([58a4e6c](58a4e6c))
* adds spellchecker workflow, and corrects misspelled words
([#559](#559))
([beac0a8](beac0a8))
* **dal:** Do not load config from env in DAL crate
([#444](#444))
([3fe1bb2](3fe1bb2))
* **en:** Implement gossip fetcher
([#371](#371))
([a49b61d](a49b61d))
* **hyperchain:** Adding prover related commands to zk stack
([#440](#440))
([580cada](580cada))
* **job-processor:** report attempts metrics
([#448](#448))
([ab31f03](ab31f03))
* **merkle tree:** Allow random-order tree recovery
([#485](#485))
([146e4cf](146e4cf))
* **witness-generator:** add logs to leaf aggregation job
([#542](#542))
([7e95a3a](7e95a3a))


### Bug Fixes

* Change no pending batches 404 error into a success response
([#279](#279))
([e8fd805](e8fd805))
* **ci:** Use the same nightly rust
([#530](#530))
([67ef133](67ef133))
* **crypto:** update shivini to switch to era-cuda
([#469](#469))
([38bb482](38bb482))
* Sync protocol version between consensus and server blocks
([#568](#568))
([56776f9](56776f9))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: Artem Makhortov <[email protected]>
github-merge-queue bot pushed a commit that referenced this pull request Dec 5, 2023
🤖 I have created a release *beep* *boop*
---


##
[10.0.0](prover-v9.1.0...prover-v10.0.0)
(2023-12-05)


### ⚠ BREAKING CHANGES

* boojum integration
([#112](#112))
* Update to protocol version 17
([#384](#384))

### Features

* Add various metrics to the Prover subsystems
([#541](#541))
([58a4e6c](58a4e6c))
* adds spellchecker workflow, and corrects misspelled words
([#559](#559))
([beac0a8](beac0a8))
* boojum integration
([#112](#112))
([e76d346](e76d346))
* **boojum:** Adding README to prover directory
([#189](#189))
([c175033](c175033))
* **config:** Extract everything not related to the env config from
zksync_config crate
([#245](#245))
([42c64e9](42c64e9))
* **core:** Split config definitions and deserialization
([#414](#414))
([c7c6b32](c7c6b32))
* **dal:** Do not load config from env in DAL crate
([#444](#444))
([3fe1bb2](3fe1bb2))
* **en:** Implement gossip fetcher
([#371](#371))
([a49b61d](a49b61d))
* **fri-prover:** In witness - panic if protocol version is not
available ([#192](#192))
([0403749](0403749))
* **hyperchain:** Adding prover related commands to zk stack
([#440](#440))
([580cada](580cada))
* **job-processor:** report attempts metrics
([#448](#448))
([ab31f03](ab31f03))
* **merkle tree:** Allow random-order tree recovery
([#485](#485))
([146e4cf](146e4cf))
* **merkle tree:** Snapshot recovery for Merkle tree
([#163](#163))
([9e20703](9e20703))
* Rewrite server binary to use `vise` metrics
([#120](#120))
([26ee1fb](26ee1fb))
* Update to protocol version 17
([#384](#384))
([ba271a5](ba271a5))
* **vm:** Move all vm versions to the one crate
([#249](#249))
([e3fb489](e3fb489))
* **witness-generator:** add logs to leaf aggregation job
([#542](#542))
([7e95a3a](7e95a3a))


### Bug Fixes

* Change no pending batches 404 error into a success response
([#279](#279))
([e8fd805](e8fd805))
* **ci:** Use the same nightly rust
([#530](#530))
([67ef133](67ef133))
* **crypto:** update deps to include circuit fixes
([#402](#402))
([4c82015](4c82015))
* **crypto:** update shivini to switch to era-cuda
([#469](#469))
([38bb482](38bb482))
* **crypto:** update snark-vk to be used in server and update args for
proof wrapping
([#240](#240))
([4a5c54c](4a5c54c))
* **docs:** Add links to setup-data keys
([#360](#360))
([1d4fe69](1d4fe69))
* **path:** update gpu prover setup data path to remove extra gpu suffix
([#454](#454))
([2e145c1](2e145c1))
* **prover-fri:** Update setup loading for node agg circuit
([#323](#323))
([d1034b0](d1034b0))
* **prover-logging:** tasks_allowed_to_finish set to true for 1 off jobs
([#227](#227))
([0fac66f](0fac66f))
* Sync protocol version between consensus and server blocks
([#568](#568))
([56776f9](56776f9))
* Update prover to use the correct storage oracle
([#446](#446))
([835dd82](835dd82))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
jrchatruc added a commit to lambdaclass/zksync-era that referenced this pull request Jan 25, 2024
* feat(merkle tree): Remove enumeration index assignment from Merkle tree (#551)

## What ❔

Since enumeration indices are now fully stored in Postgres, it makes
sense to not duplicate their assignment in the Merkle tree. Instead, the
tree could take enum indices as inputs.

## Why ❔

This allows simplifying tree logic and unify "normal" L1 batch
processing and tree recovery. (This unification is not a part of this
PR; it'll be implemented separately.)

## Checklist

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.

* feat(en): Support arbitrary genesis block for external nodes (#537)

## What ❔

Support non-zero genesis block specified in executor configuration.
Check whether this block exists on initialization; validate its
correspondence if it does, and persist consensus fields if it doesn't.

## Why ❔

This is necessary to support gossip-based syncing in practice; we likely
won't back-sign all blocks in all envs.

## Checklist

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.

* fix(witness_generator): Disable BWIP dependency (#573)

This revert is done to facilitate boojum upgrade on mainnet2. Without
this, old provers would be halt and boojum upgrade could take longer
than anticipated.

`waiting_for_artifacts` forced witness to wait on BWIP run. `queued`
makes them run instantly.

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

---------

Signed-off-by: Danil <[email protected]>
Co-authored-by: Danil <[email protected]>

* chore(main): release core 18.4.0 (#560)

:robot: I have created a release *beep* *boop*
---


##
[18.4.0](https://github.com/matter-labs/zksync-era/compare/core-v18.3.1...core-v18.4.0)
(2023-12-01)


### Features

* adds spellchecker workflow, and corrects misspelled words
([#559](https://github.com/matter-labs/zksync-era/issues/559))
([beac0a8](https://github.com/matter-labs/zksync-era/commit/beac0a85bb1535b05c395057171f197cd976bf82))
* **en:** Support arbitrary genesis block for external nodes
([#537](https://github.com/matter-labs/zksync-era/issues/537))
([15d7eaf](https://github.com/matter-labs/zksync-era/commit/15d7eaf872e222338810243865cec9dff7f6e799))
* **merkle tree:** Remove enumeration index assignment from Merkle tree
([#551](https://github.com/matter-labs/zksync-era/issues/551))
([e2c1b20](https://github.com/matter-labs/zksync-era/commit/e2c1b20e361e6ee2f5ac69cefe75d9c5575eb2f7))
* Restore commitment test in Boojum integration
([#539](https://github.com/matter-labs/zksync-era/issues/539))
([06f510d](https://github.com/matter-labs/zksync-era/commit/06f510d00f855ddafaebb504f7ea799700221072))


### Bug Fixes

* Change no pending batches 404 error into a success response
([#279](https://github.com/matter-labs/zksync-era/issues/279))
([e8fd805](https://github.com/matter-labs/zksync-era/commit/e8fd805c8be7980de7676bca87cfc2d445aab9e1))
* **vm:** Expose additional types and traits
([#563](https://github.com/matter-labs/zksync-era/issues/563))
([bd268ac](https://github.com/matter-labs/zksync-era/commit/bd268ac02bc3530c1d3247cb9496c3e13c2e52d9))
* **witness_generator:** Disable BWIP dependency
([#573](https://github.com/matter-labs/zksync-era/issues/573))
([e05d955](https://github.com/matter-labs/zksync-era/commit/e05d955036c76a29f9b6e900872c69e20278e045))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

* ci: Runs spellcheck in merge queue. (#574)

## What ❔

Runs spellcheck in merge queue.
<!-- What are the changes this PR brings about? -->
<!-- Example: This PR adds a PR template to the repo. -->
<!-- (For bigger PRs adding more context is appreciated) -->

## Why ❔

<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

* chore: fix typo (#575)

## What ❔

- fix typo

## Why ❔

- fix typo

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

---------

Co-authored-by: Igor Aleksanov <[email protected]>

* chore: fix typos in document (#577)

## What ❔

- fixed typo

## Why ❔

fix typos in document

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

---------

Co-authored-by: Igor Aleksanov <[email protected]>

* chore: Fix typos (#567)

Hi, I have just resolve conflict #432

Co-authored-by: Igor Aleksanov <[email protected]>

* feat: Add metric to CallTracer for calculating maximum depth of the calls (#535)

## What ❔

Add metric to CallTracer for calculating maximum depth of the calls

## Why ❔

We need to know what our limits are.

## Checklist

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.

* feat: Add various metrics to the Prover subsystems (#541)

## What ❔

1. Add various metrics to the Prover subsystems, especially:
* oldest block, that wasn't sent to
prover(`fri_prover.oldest_unprocessed_block`)
* oldest block, that didn't go through basic/leaf/node aggregation
levels (`fri_prover.oldest_unprocessed_block_by_round`)
* how much time is spent on waiting for available prover to send data to
(`prover_fri_witness_vector_generator.prover_waiting_time)
* count for attempts to send data to prover
(`prover_fri_witness_vector_generator.prover_attempts_count`)
2. Refactor metrics in prover to use vise.

## Why ❔

We have some metric coverage on the prover subsystem, but it's
incomplete.

## Checklist

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.

* chore: fix wrong line  (#592)

## What ❔

fix wrong line 

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

* chore(docs): fixed docs typo (#588)

## What ❔

- Hello, fixed typo
## Why ❔

- fixed typo

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

---------

Co-authored-by: Roman Brodetski <[email protected]>

* chore(docs):  fix typos in document (#589)

## What ❔

Hello, I corrected the typo.

## Why ❔

- fixed typo

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

---------

Co-authored-by: Igor Aleksanov <[email protected]>

* chore: fix typo (#587)

## What ❔

fixed typos

## Why ❔

fixed typos

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

---------

Co-authored-by: Igor Aleksanov <[email protected]>

* chore(docs): fix broken link (#590)

## What ❔

fixed broken link


## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

---------

Co-authored-by: Igor Aleksanov <[email protected]>

* feat: faster and less noisy zk fmt (#513)

## What ❔

I've added caching to prettier and changed so that noisy output about
changed files is redirected to /dev/null. `zk fmt` is 3 times faster
after those changes

## Why ❔

`zk fmt` output was too verbose and we didn't use cache

## Checklist

- [X] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [X] Code has been formatted via `zk fmt` and `zk lint`.

* chore: the errors in the document have been correct (#583)

## What ❔

the errors in the document have been correct

## Why ❔

the errors in the document have been correct

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

---------

Co-authored-by: Igor Aleksanov <[email protected]>

* chore(docs): the errors in the document have been corrected. (#461)

## What ❔

- the errors in the document have been corrected.

## Why ❔

- the errors in the document have been corrected.

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.

---------

Co-authored-by: perekopskiy <[email protected]>

* chore: update document (#601)

## What ❔

- update document


## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

---------

Co-authored-by: Igor Aleksanov <[email protected]>
Co-authored-by: perekopskiy <[email protected]>

* chore: fixed typos in documentation (#603)

## What ❔

- fixed typos in documentation

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

---------

Co-authored-by: perekopskiy <[email protected]>

* chore: remove incorrect branch prompts (#594)

## What ❔

remove incorrect branch prompts

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

---------

Co-authored-by: Igor Aleksanov <[email protected]>

* fix: Sync protocol version between consensus and server blocks (#568)

## What ❔

Aligns the protocol version for consensus blocks with that of
`SyncBlock`s.

## Why ❔

Required for gossip-based block syncing to work correctly.

## Checklist

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

* chore(main): release core 18.5.0 (#593)

:robot: I have created a release *beep* *boop*
---


##
[18.5.0](https://github.com/matter-labs/zksync-era/compare/core-v18.4.0...core-v18.5.0)
(2023-12-05)


### Features

* Add metric to CallTracer for calculating maximum depth of the calls
([#535](https://github.com/matter-labs/zksync-era/issues/535))
([19c84ce](https://github.com/matter-labs/zksync-era/commit/19c84ce624d53735133fa3b12c7f980e8c14260d))
* Add various metrics to the Prover subsystems
([#541](https://github.com/matter-labs/zksync-era/issues/541))
([58a4e6c](https://github.com/matter-labs/zksync-era/commit/58a4e6c4c22bd7f002ede1c6def0dc260706185e))


### Bug Fixes

* Sync protocol version between consensus and server blocks
([#568](https://github.com/matter-labs/zksync-era/issues/568))
([56776f9](https://github.com/matter-labs/zksync-era/commit/56776f929f547b1a91c5b70f89e87ef7dc25c65a))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

* chore: fix link (#576)

## What ❔

fix link

## Why ❔

- fix link

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

---------

Co-authored-by: Igor Aleksanov <[email protected]>
Co-authored-by: perekopskiy <[email protected]>

* chore(main): release prover 9.1.0 (#460)

:robot: I have created a release *beep* *boop*
---


##
[9.1.0](https://github.com/matter-labs/zksync-era/compare/prover-v9.0.0...prover-v9.1.0)
(2023-12-05)


### Features

* Add various metrics to the Prover subsystems
([#541](https://github.com/matter-labs/zksync-era/issues/541))
([58a4e6c](https://github.com/matter-labs/zksync-era/commit/58a4e6c4c22bd7f002ede1c6def0dc260706185e))
* adds spellchecker workflow, and corrects misspelled words
([#559](https://github.com/matter-labs/zksync-era/issues/559))
([beac0a8](https://github.com/matter-labs/zksync-era/commit/beac0a85bb1535b05c395057171f197cd976bf82))
* **dal:** Do not load config from env in DAL crate
([#444](https://github.com/matter-labs/zksync-era/issues/444))
([3fe1bb2](https://github.com/matter-labs/zksync-era/commit/3fe1bb21f8d33557353f447811ca86c60f1fe51a))
* **en:** Implement gossip fetcher
([#371](https://github.com/matter-labs/zksync-era/issues/371))
([a49b61d](https://github.com/matter-labs/zksync-era/commit/a49b61d7769f9dd7b4cbc4905f8f8a23abfb541c))
* **hyperchain:** Adding prover related commands to zk stack
([#440](https://github.com/matter-labs/zksync-era/issues/440))
([580cada](https://github.com/matter-labs/zksync-era/commit/580cada003bdfe2fff686a1fc3ce001b4959aa4d))
* **job-processor:** report attempts metrics
([#448](https://github.com/matter-labs/zksync-era/issues/448))
([ab31f03](https://github.com/matter-labs/zksync-era/commit/ab31f031dfcaa7ddf296786ddccb78e8edd2d3c5))
* **merkle tree:** Allow random-order tree recovery
([#485](https://github.com/matter-labs/zksync-era/issues/485))
([146e4cf](https://github.com/matter-labs/zksync-era/commit/146e4cf2f8d890ff0a8d33229e224442e14be437))
* **witness-generator:** add logs to leaf aggregation job
([#542](https://github.com/matter-labs/zksync-era/issues/542))
([7e95a3a](https://github.com/matter-labs/zksync-era/commit/7e95a3a66ea48be7b6059d34630e22c503399bdf))


### Bug Fixes

* Change no pending batches 404 error into a success response
([#279](https://github.com/matter-labs/zksync-era/issues/279))
([e8fd805](https://github.com/matter-labs/zksync-era/commit/e8fd805c8be7980de7676bca87cfc2d445aab9e1))
* **ci:** Use the same nightly rust
([#530](https://github.com/matter-labs/zksync-era/issues/530))
([67ef133](https://github.com/matter-labs/zksync-era/commit/67ef1339d42786efbeb83c22fac99f3bf5dd4380))
* **crypto:** update shivini to switch to era-cuda
([#469](https://github.com/matter-labs/zksync-era/issues/469))
([38bb482](https://github.com/matter-labs/zksync-era/commit/38bb4823c7b5e0e651d9f531feede66c24afd19f))
* Sync protocol version between consensus and server blocks
([#568](https://github.com/matter-labs/zksync-era/issues/568))
([56776f9](https://github.com/matter-labs/zksync-era/commit/56776f929f547b1a91c5b70f89e87ef7dc25c65a))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: Artem Makhortov <[email protected]>

* chore(main): release prover 10.0.0 (#608)

:robot: I have created a release *beep* *boop*
---


##
[10.0.0](https://github.com/matter-labs/zksync-era/compare/prover-v9.1.0...prover-v10.0.0)
(2023-12-05)


### ⚠ BREAKING CHANGES

* boojum integration
([#112](https://github.com/matter-labs/zksync-era/issues/112))
* Update to protocol version 17
([#384](https://github.com/matter-labs/zksync-era/issues/384))

### Features

* Add various metrics to the Prover subsystems
([#541](https://github.com/matter-labs/zksync-era/issues/541))
([58a4e6c](https://github.com/matter-labs/zksync-era/commit/58a4e6c4c22bd7f002ede1c6def0dc260706185e))
* adds spellchecker workflow, and corrects misspelled words
([#559](https://github.com/matter-labs/zksync-era/issues/559))
([beac0a8](https://github.com/matter-labs/zksync-era/commit/beac0a85bb1535b05c395057171f197cd976bf82))
* boojum integration
([#112](https://github.com/matter-labs/zksync-era/issues/112))
([e76d346](https://github.com/matter-labs/zksync-era/commit/e76d346d02ded771dea380aa8240da32119d7198))
* **boojum:** Adding README to prover directory
([#189](https://github.com/matter-labs/zksync-era/issues/189))
([c175033](https://github.com/matter-labs/zksync-era/commit/c175033b48a8da4969d88b6850dd0247c4004794))
* **config:** Extract everything not related to the env config from
zksync_config crate
([#245](https://github.com/matter-labs/zksync-era/issues/245))
([42c64e9](https://github.com/matter-labs/zksync-era/commit/42c64e91e13b6b37619f1459f927fa046ef01097))
* **core:** Split config definitions and deserialization
([#414](https://github.com/matter-labs/zksync-era/issues/414))
([c7c6b32](https://github.com/matter-labs/zksync-era/commit/c7c6b321a63dbcc7f1af045aa7416e697beab08f))
* **dal:** Do not load config from env in DAL crate
([#444](https://github.com/matter-labs/zksync-era/issues/444))
([3fe1bb2](https://github.com/matter-labs/zksync-era/commit/3fe1bb21f8d33557353f447811ca86c60f1fe51a))
* **en:** Implement gossip fetcher
([#371](https://github.com/matter-labs/zksync-era/issues/371))
([a49b61d](https://github.com/matter-labs/zksync-era/commit/a49b61d7769f9dd7b4cbc4905f8f8a23abfb541c))
* **fri-prover:** In witness - panic if protocol version is not
available ([#192](https://github.com/matter-labs/zksync-era/issues/192))
([0403749](https://github.com/matter-labs/zksync-era/commit/040374900656c854a7b9de32e5dbaf47c1c47889))
* **hyperchain:** Adding prover related commands to zk stack
([#440](https://github.com/matter-labs/zksync-era/issues/440))
([580cada](https://github.com/matter-labs/zksync-era/commit/580cada003bdfe2fff686a1fc3ce001b4959aa4d))
* **job-processor:** report attempts metrics
([#448](https://github.com/matter-labs/zksync-era/issues/448))
([ab31f03](https://github.com/matter-labs/zksync-era/commit/ab31f031dfcaa7ddf296786ddccb78e8edd2d3c5))
* **merkle tree:** Allow random-order tree recovery
([#485](https://github.com/matter-labs/zksync-era/issues/485))
([146e4cf](https://github.com/matter-labs/zksync-era/commit/146e4cf2f8d890ff0a8d33229e224442e14be437))
* **merkle tree:** Snapshot recovery for Merkle tree
([#163](https://github.com/matter-labs/zksync-era/issues/163))
([9e20703](https://github.com/matter-labs/zksync-era/commit/9e2070380e6720d84563a14a2246fc18fdb1f8f9))
* Rewrite server binary to use `vise` metrics
([#120](https://github.com/matter-labs/zksync-era/issues/120))
([26ee1fb](https://github.com/matter-labs/zksync-era/commit/26ee1fbb16cbd7c4fad334cbc6804e7d779029b6))
* Update to protocol version 17
([#384](https://github.com/matter-labs/zksync-era/issues/384))
([ba271a5](https://github.com/matter-labs/zksync-era/commit/ba271a5f34d64d04c0135b8811685b80f26a8c32))
* **vm:** Move all vm versions to the one crate
([#249](https://github.com/matter-labs/zksync-era/issues/249))
([e3fb489](https://github.com/matter-labs/zksync-era/commit/e3fb4894d08aa98a84e64eaa95b51001055cf911))
* **witness-generator:** add logs to leaf aggregation job
([#542](https://github.com/matter-labs/zksync-era/issues/542))
([7e95a3a](https://github.com/matter-labs/zksync-era/commit/7e95a3a66ea48be7b6059d34630e22c503399bdf))


### Bug Fixes

* Change no pending batches 404 error into a success response
([#279](https://github.com/matter-labs/zksync-era/issues/279))
([e8fd805](https://github.com/matter-labs/zksync-era/commit/e8fd805c8be7980de7676bca87cfc2d445aab9e1))
* **ci:** Use the same nightly rust
([#530](https://github.com/matter-labs/zksync-era/issues/530))
([67ef133](https://github.com/matter-labs/zksync-era/commit/67ef1339d42786efbeb83c22fac99f3bf5dd4380))
* **crypto:** update deps to include circuit fixes
([#402](https://github.com/matter-labs/zksync-era/issues/402))
([4c82015](https://github.com/matter-labs/zksync-era/commit/4c820150714dfb01c304c43e27f217f17deba449))
* **crypto:** update shivini to switch to era-cuda
([#469](https://github.com/matter-labs/zksync-era/issues/469))
([38bb482](https://github.com/matter-labs/zksync-era/commit/38bb4823c7b5e0e651d9f531feede66c24afd19f))
* **crypto:** update snark-vk to be used in server and update args for
proof wrapping
([#240](https://github.com/matter-labs/zksync-era/issues/240))
([4a5c54c](https://github.com/matter-labs/zksync-era/commit/4a5c54c48bbc100c29fa719c4b1dc3535743003d))
* **docs:** Add links to setup-data keys
([#360](https://github.com/matter-labs/zksync-era/issues/360))
([1d4fe69](https://github.com/matter-labs/zksync-era/commit/1d4fe697e4e98a8e64642cde4fe202338ce5ec61))
* **path:** update gpu prover setup data path to remove extra gpu suffix
([#454](https://github.com/matter-labs/zksync-era/issues/454))
([2e145c1](https://github.com/matter-labs/zksync-era/commit/2e145c192b348b2756acf61fac5bfe0ca5a6575f))
* **prover-fri:** Update setup loading for node agg circuit
([#323](https://github.com/matter-labs/zksync-era/issues/323))
([d1034b0](https://github.com/matter-labs/zksync-era/commit/d1034b05754219b603508ef79c114d908c94c1e9))
* **prover-logging:** tasks_allowed_to_finish set to true for 1 off jobs
([#227](https://github.com/matter-labs/zksync-era/issues/227))
([0fac66f](https://github.com/matter-labs/zksync-era/commit/0fac66f5ff86cc801ea0bb6f9272cb397cd03a95))
* Sync protocol version between consensus and server blocks
([#568](https://github.com/matter-labs/zksync-era/issues/568))
([56776f9](https://github.com/matter-labs/zksync-era/commit/56776f929f547b1a91c5b70f89e87ef7dc25c65a))
* Update prover to use the correct storage oracle
([#446](https://github.com/matter-labs/zksync-era/issues/446))
([835dd82](https://github.com/matter-labs/zksync-era/commit/835dd828ef5610a446ec8c456e4df1def0e213ab))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

* refactor: Removed protobuf encoding from zksync_types (#562)

## What ❔

Removed protobuf encoding from zksync_types.

## Why ❔

To make zksync_types have less dependencies.

* fix: use powers array in plonkSetup function (#508)

## What ❔

This PR modifies the `plonkSetup` function in `run.ts` to use the
`powers` array when downloading key files.

## Why ❔

Previously, the function forget to use the argument values `powers`.
Now, it will download keys for any powers specified in the `powers`
array.

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [NA] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.

---------

Co-authored-by: Igor Aleksanov <[email protected]>

* fix: Fix database connections in house keeper (#610)

## What ❔

Use correct connections for databases in house keeper.

## Why ❔

Databases are divided in 2 on mainnet and testnet

## Checklist

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

* feat(contract-verifier): Support verification for zksolc v1.3.17 (#606)

## What ❔

Adds support for zksolc v1.3.17 to contract-verifier.

## Why ❔

Contract-verifier should support latest version

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

* chore(main): release core 18.6.0 (#613)

:robot: I have created a release *beep* *boop*
---


##
[18.6.0](https://github.com/matter-labs/zksync-era/compare/core-v18.5.0...core-v18.6.0)
(2023-12-05)


### Features

* **contract-verifier:** Support verification for zksolc v1.3.17
([#606](https://github.com/matter-labs/zksync-era/issues/606))
([b65fedd](https://github.com/matter-labs/zksync-era/commit/b65fedd6894497a4c9fbf38d558ccfaca535d1d2))


### Bug Fixes

* Fix database connections in house keeper
([#610](https://github.com/matter-labs/zksync-era/issues/610))
([aeaaecb](https://github.com/matter-labs/zksync-era/commit/aeaaecb54b6bd3f173727531418dc242357b2aee))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

* chore: Mainnet upgrade calldata (#564)

## What ❔

Includes mainnet upgrade preparation as well as some minor fixes for the
upgrade tool

## Why ❔

<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.

---------

Co-authored-by: koloz <[email protected]>

* chore: Update generated Prover FRI CPU setup-data keys from branch main (#609)

"Update generated Prover FRI CPU setup-data keys from branch main"

* perf(external-node): Use async miniblock sealing in external IO (#611)

## What ❔

External IO uses async miniblock sealing. 

## Why ❔

Execution of transactions and miniblock sealing (writing data to
postgres) happen in parallel so the perfomance is better.

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

* chore: fix document path (#615)

## What ❔

fix document path

## Why ❔

<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

---------

Co-authored-by: Igor Aleksanov <[email protected]>

* chore: Remove era-reviewers from codeowners (#618)

## What ❔

Removes era-reviewers group from codeowners.

## Why ❔

- Too noisy.
- We have internal processes for that anyways.

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

* chore(main): release core 18.6.1 (#616)

:robot: I have created a release *beep* *boop*
---


##
[18.6.1](https://github.com/matter-labs/zksync-era/compare/core-v18.6.0...core-v18.6.1)
(2023-12-06)


### Performance Improvements

* **external-node:** Use async miniblock sealing in external IO
([#611](https://github.com/matter-labs/zksync-era/issues/611))
([5cf7210](https://github.com/matter-labs/zksync-era/commit/5cf7210dc77bb615944352f23ed39fad324b914f))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

* feat(hyperchain-wizard): zkStack CLI GPU support (#612)

## What ❔

Support for creating zk hyperchain via zk cli with GPU-based provers

## Why ❔


## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [X] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

---------

Co-authored-by: Igor Borodin <[email protected]>

* fix: Cursor not moving correctly after poll in `get_filter_changes` (#546)

## What ❔

When polling filter changes, add 1 to actual from_block value

## Why ❔

Otherwise, last block that was included in poll will be included to the
next one.

## Checklist

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.

---------

Co-authored-by: Fedor Sakharov <[email protected]>

* fix: update google cloud dependencies that do not depend on rsa (#622)

## What ❔

This PR updates the dependencies of `google-cloud-storage` and
`google-cloud-auth`. The changes are as follows:

- From google-cloud-storage = "0.12.0" to google-cloud-storage =
"0.15.0"
- From google-cloud-auth = "0.11.0" to google-cloud-auth = "0.13.0"

Relevant google-cloud changes:
https://github.com/yoshidan/google-cloud-rust/pull/217

## Why ❔

<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

The primary reason for these updates is to address a security
vulnerability associated with the `rsa` crate, as indicated by a recent
`cargo-deny` check. The vulnerability (Marvin Attack, RUSTSEC-2023-0071)
was detected in rsa v0.6.1, which is a dependency of
`google-cloud-storage v0.12.0`. By updating to `google-cloud-storage
v0.15.0`, we eliminate the use of the `rsa` crate, as the newer version
of `google-cloud-storage` does not depend on it. Similarly,
`google-cloud-auth` is updated for compatibility.

Cargo deny output:

```
error[vulnerability]: Marvin Attack: potential key recovery through timing sidechannels
    ┌─ /Users/dustinbrickwood/Documents/dev/dut/forks/foundry-zksync/Cargo.lock:759:1
    │
759 │ rsa 0.6.1 registry+https://github.com/rust-lang/crates.io-index
    │ --------------------------------------------------------------- security vulnerability detected
    │
    = ID: RUSTSEC-2023-0071
    = Advisory: https://rustsec.org/advisories/RUSTSEC-2023-0071
    = ### Impact
      Due to a non-constant-time implementation, information about the private key is leaked through timing information which is observable over the network. An attacker may be able to use that information to recover the key.

      ### Patches
      No patch is yet available, however work is underway to migrate to a fully constant-time implementation.

      ### Workarounds
      The only currently available workaround is to avoid using the `rsa` crate in settings where attackers are able to observe timing information, e.g. local use on a non-compromised computer is fine.

      ### References
      This vulnerability was discovered as part of the "[Marvin Attack]", which revealed several implementations of RSA including OpenSSL had not properly mitigated timing sidechannel attacks.

      [Marvin Attack]: https://people.redhat.com/~hkario/marvin/
    = Announcement: https://github.com/RustCrypto/RSA/issues/19#issuecomment-1822995643
    = Solution: No safe upgrade is available!
    = rsa v0.6.1
      └── google-cloud-storage v0.12.0
          └── zksync_object_store v0.1.0
              ├── zksync_core v0.1.0
              │   └── era_test_node v0.1.0-alpha.12
              │       └── era_revm v0.0.1-alpha
              │           ├── foundry-common v0.2.0
              │           │   ├── anvil v0.2.0
              │           │   │   ├── (dev) forge v0.2.0
              │           │   │   └── (dev) zkforge v0.2.0
              │           │   ├── cast v0.2.0
              │           │   ├── chisel v0.2.0
              │           │   ├── forge v0.2.0 (*)
              │           │   ├── foundry-cli v0.2.0
              │           │   │   ├── cast v0.2.0 (*)
              │           │   │   ├── chisel v0.2.0 (*)
              │           │   │   ├── forge v0.2.0 (*)
              │           │   │   ├── zkcast v0.2.0
              │           │   │   │   └── zkforge v0.2.0 (*)
              │           │   │   └── zkforge v0.2.0 (*)
              │           │   ├── foundry-debugger v0.2.0
              │           │   │   ├── forge v0.2.0 (*)
              │           │   │   ├── foundry-cli v0.2.0 (*)
              │           │   │   └── zkforge v0.2.0 (*)
              │           │   ├── foundry-evm v0.2.0
              │           │   │   ├── anvil v0.2.0 (*)
              │           │   │   ├── anvil-core v0.2.0
              │           │   │   │   └── anvil v0.2.0 (*)
              │           │   │   ├── cast v0.2.0 (*)
              │           │   │   ├── chisel v0.2.0 (*)
              │           │   │   ├── forge v0.2.0 (*)
              │           │   │   ├── foundry-cli v0.2.0 (*)
              │           │   │   ├── foundry-debugger v0.2.0 (*)
              │           │   │   ├── zkcast v0.2.0 (*)
              │           │   │   └── zkforge v0.2.0 (*)
              │           │   ├── foundry-test-utils v0.2.0
              │           │   │   ├── (dev) cast v0.2.0 (*)
              │           │   │   ├── (dev) forge v0.2.0 (*)
              │           │   │   ├── (dev) zkcast v0.2.0 (*)
              │           │   │   └── (dev) zkforge v0.2.0 (*)
              │           │   ├── (dev) foundry-utils v0.2.0
              │           │   │   ├── anvil v0.2.0 (*)
              │           │   │   ├── anvil-core v0.2.0 (*)
              │           │   │   ├── cast v0.2.0 (*)
              │           │   │   ├── chisel v0.2.0 (*)
              │           │   │   ├── forge v0.2.0 (*)
              │           │   │   ├── forge-doc v0.2.0
              │           │   │   │   ├── forge v0.2.0 (*)
              │           │   │   │   └── zkforge v0.2.0 (*)
              │           │   │   ├── foundry-cli v0.2.0 (*)
              │           │   │   ├── foundry-debugger v0.2.0 (*)
              │           │   │   ├── (dev) foundry-evm v0.2.0 (*)
              │           │   │   ├── foundry-test-utils v0.2.0 (*)
              │           │   │   ├── zkcast v0.2.0 (*)
              │           │   │   └── zkforge v0.2.0 (*)
              │           │   ├── zkcast v0.2.0 (*)
              │           │   └── zkforge v0.2.0 (*)
              │           └── foundry-evm v0.2.0 (*)
              └── zksync_prover_utils v0.1.0
                  ├── zksync_core v0.1.0 (*)
                  └── zksync_verification_key_generator_and_server v0.1.0
                      └── zksync_core v0.1.0 (*)
```

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

* docs: Include command to create rich L2 wallets. (#569)

## What ❔

Improve documentation by including the command to create rich L2
wallets.

## Why ❔

Save other people time figuring out the exact invocation.

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

* chore: Enforce uniform import structure (#617)

## What ❔

...using `zk fmt` command by suppling relevant command-line args to
rustfmt. These args work on stable Rust (at least for now) despite being
unstable.

## Why ❔

More structured imports are easier to read.

## Checklist

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

* fix(job-processor): `max_attepts_reached` metric (#626)

## What ❔

`max_attepts_reached` metric is now reported on job start rather
failure. With this change metric will be reported not only if last
attempt failed but also if it got stuck/stopped/etc.

## Why ❔

Reporting `max_attepts_reached` metric for all cases.

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

* chore(vm): Expose more pubs and make inmemory clonable (#632)

## What ❔

Expose AppFramestack public and make InMemoryStorage Clonable.

## Why ❔

For supporting era-test-node

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

---------

Signed-off-by: Danil <[email protected]>

* chore: remove old witness generator (#619)

* fix: improve docs repositories (#570)

## What ❔

* Add new/missing zkSync repositories
* Add missing descriptions
* Remove deprecated repositories

## Why ❔

To make the list up-to-date

## Checklist

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

* docs(setup): Added TL;DR instructions for new zkstack setup (#621)

## What ❔

* Added a TL;DR set of instructions needed to setup the system to run
the zk stack from scratch.


## Why ❔
* To have a list of commands in one place.

* chore: upgrades local test network to cancun+deneb compatible one (#580)

## What ❔

Upgrades local testnet to `Cancun+Deneb` compatible one. So far:

Cancun gets enabled:

```
2023-12-01 21:57:49 INFO [12-01|20:57:49.152] Merge configured: 
2023-12-01 21:57:49 INFO [12-01|20:57:49.152]  - Hard-fork specification:    https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/paris.md 
2023-12-01 21:57:49 INFO [12-01|20:57:49.152]  - Network known to be merged: true 
2023-12-01 21:57:49 INFO [12-01|20:57:49.152]  - Total terminal difficulty:  0 
2023-12-01 21:57:49 INFO [12-01|20:57:49.152]  
2023-12-01 21:57:49 INFO [12-01|20:57:49.152] Post-Merge hard forks (timestamp based): 
2023-12-01 21:57:49 INFO [12-01|20:57:49.152]  - Shanghai:                    @1701464272 (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/shanghai.md) 
2023-12-01 21:57:49 INFO [12-01|20:57:49.152]  - Cancun:                      @1701464272 
```

New network has been built into CI workflows.


## Why ❔

<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

* chore(zk init): Removed plonk setup (#638)

## What ❔

* Removed plonk setup (where we downloaded the CRS keys)

## Why ❔

* It is needed only for prover setup (and then this is handled within
the prover_setup.ts directly)
* it will save a lot of network bandwidth and time during zk stack setup

* chore(CI): Speeding up docker builds in CI (#640)

## What ❔

* Removed zk contract compilation that was run before prover docker
builds
* Removed docker builds for old prover and old circuit_synthesizer
* Downloading CRS file only for the snark wrapper / compressor job

## Why ❔

* to speed up docker CI that runs on every PR

* chore(zk): finishing migration to docker compose (#646)

## What ❔

* replaced docker-compose with docker compose 

## Why ❔

* we moved to use docker compose (instead of docker-compose) - which is
a newer version

* chore(ci): Pre-download compilers, as a workaround for old hardhat plugins (#645)

## What ❔

Add hack way to pre-download needed compileres

## Why ❔

Hardhat plugins, which we are using right now, are pretty often rate
limited by github for downloading compilers, due to unoptimized download
procedure. This issue was fixed in later versions of plugins, but we
can't update _some_ of the hardhat plugins due to changes in deps for
them. This pre-download will help us till the time we can update them.

## Checklist


- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

* feat(en): Remove `SyncBlock.root_hash` (#633)

## What ❔

Removes `root_hash` field from `SyncBlock`.

## Why ❔

It's not used anywhere, set to a dummy value (miniblock hash) and
doesn't make sense in general (state hashes are computed on L1 batch
level, not miniblock level).

## Checklist

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

* docs: New protocol specification (#641)

## What ❔

New docs, focusing on protocol specs.  

Note: don't merge yet, waiting for review.

## Why ❔

We want to have good docs so people can understand the zkVM and its
benefits.

## Checklist

- [ X] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ X] Tests for the changes have been added / updated.
- [ X] Documentation comments have been added / updated.
- [ X] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

---------

Co-authored-by: kelemeno <[email protected]>
Co-authored-by: MexicanAce <[email protected]>
Co-authored-by: Jack Hamer <[email protected]>
Co-authored-by: Marcin M <[email protected]>
Co-authored-by: Fedor Sakharov <[email protected]>

* feat(zk tool): Added yarn & directory checks (#188)

# What ❔

* zk will now print a warning if you run it from outside of $ZKSYNC_HOME
directory
* it will also warn you if your yarn has a wrong version

## Why ❔

* we use ZKSYNC_HOME to figure out which binaries to use, so if you are
working on couple versions of era in multiple different directories, and
forget to update ZKSYNC_HOME, then you might be pushing wrong binaries
without knowing.
* if your version of yarn is incorrect, you're going to get a lot of
strange typescript compilation errors along the way.

These are only printed as warnings, they don't prevent you from running
the tool.


![image](https://github.com/matter-labs/zksync-era/assets/128217157/e8186e36-953d-4307-9449-048bdd589321)

* fix: follow up metrics fixes (#648)

## What ❔

Fixes for previous metrics PRs.

* Fix oldest unproved block by round (we have `manually_skipped` status
in DB, is this correct?), SQL query and number of rounds
* Rename some API server metrics
* May filter count work incorrect?

## Why ❔

For metrics to work correctly.

## Checklist

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

* chore(ci): Pre-download one more version of viper compiler (#652)

## What ❔

Pre-download one more version of viper compiler

## Why ❔

Workaround for rate limiting

## Checklist

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

* feat(en): Check block hash correspondence (#572)

## What ❔

Checks that the block hash received by EN corresponds to the locally
computed block hash. The check occurs reasonably early (i.e., before the
block is executed).

## Why ❔

Allows to ensure that there's nothing fundamentally wrong with EN
implementation.

## Checklist

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

* fix: fix docs error (#635)

## What ❔

fix docs error

## Why ❔

<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

---------

Co-authored-by: Igor Aleksanov <[email protected]>
Co-authored-by: Igor Borodin <[email protected]>
Co-authored-by: AnastasiiaVashchuk <[email protected]>

* chore(core/lib): typo fix (#624)

few minor typo fix.

3 comments, 1 error message

---------

Co-authored-by: AnastasiiaVashchuk <[email protected]>

* chore: remove tslint related comments (#636)

## What ❔

This pull request aims to remove TSLint-related comments and
configurations from the codebase.

## Why ❔

The removal is necessary because the repository has transitioned to
ESLint, TSLint-related comments and configurations obsolete. This
cleanup ensures a more streamlined and consistent codebase, aligning
with the migration to ESLint for improved code quality and
maintainability.

## Checklist

- [X] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [X] Tests for the changes have been added / updated.
- [X] Documentation comments have been added / updated.
- [X] Code has been formatted via `zk fmt` and `zk lint`.
- [X] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

---------

Co-authored-by: AnastasiiaVashchuk <[email protected]>

* feat: Snapshot Creator (#498)

## What ❔
Snapshot creator is small command line tool for creating a snapshot of
zkSync node for EN node to be able to initialize
to a certain L1 Batch.

Snapshots do not contain full transactions history, but rather a minimal
subset of information needed to bootstrap EN
node.

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [X] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [X] Tests for the changes have been added / updated.
- [X] Documentation comments have been added / updated.
- [X] Code has been formatted via `zk fmt` and `zk lint`.

* chore(ci): Pre-download more versions of compilers + less verbose (#653)

## What ❔

Pre-download more versions of compilers

## Checklist

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

* feat(contract-verifier): Add zksolc v1.3.18 (#654)

## What ❔

Adds zksolc v1.3.18 to contract verifier

## Why ❔

Adds zksolc v1.3.18 to contract verifier

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

* fix: Follow up metrics fixes vol.2 (#656)

## What ❔

Fix oldest not generated batch query

## Why ❔

For metrics to work correctly

## Checklist

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [x] Code has been formatted via `zk fmt` and `zk lint`.
- [x] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

* fix: removing sqlx check from pre-push (#658)

Removing this check for now, I will re-add is as an opt-in check,
probably with other checks bundled in. It's best not to require built
project on every push.

* chore(main): release core 18.7.0 (#620)

:robot: I have created a release *beep* *boop*
---


##
[18.7.0](https://github.com/matter-labs/zksync-era/compare/core-v18.6.1...core-v18.7.0)
(2023-12-12)


### Features

* **contract-verifier:** Add zksolc v1.3.18
([#654](https://github.com/matter-labs/zksync-era/issues/654))
([77f91fe](https://github.com/matter-labs/zksync-era/commit/77f91fe253a0876e56de4aee47071fe249386fc7))
* **en:** Check block hash correspondence
([#572](https://github.com/matter-labs/zksync-era/issues/572))
([28f5642](https://github.com/matter-labs/zksync-era/commit/28f5642c35800997879bc549fca9e960c4516d21))
* **en:** Remove `SyncBlock.root_hash`
([#633](https://github.com/matter-labs/zksync-era/issues/633))
([d4cc6e5](https://github.com/matter-labs/zksync-era/commit/d4cc6e564642b4c49ef4a546cd1c86821327683c))
* Snapshot Creator
([#498](https://github.com/matter-labs/zksync-era/issues/498))
([270edee](https://github.com/matter-labs/zksync-era/commit/270edee34402ecbd1761bc1fca559ef2205f71e8))


### Bug Fixes

* Cursor not moving correctly after poll in `get_filter_changes`
([#546](https://github.com/matter-labs/zksync-era/issues/546))
([ec5907b](https://github.com/matter-labs/zksync-era/commit/ec5907b70ff7d868a05b685a1641d96dc4fa9d69))
* fix docs error
([#635](https://github.com/matter-labs/zksync-era/issues/635))
([883c128](https://github.com/matter-labs/zksync-era/commit/883c1282f7771fb16a41d45391b74243021271e3))
* follow up metrics fixes
([#648](https://github.com/matter-labs/zksync-era/issues/648))
([a317c7a](https://github.com/matter-labs/zksync-era/commit/a317c7ab68219cb376d08c8d1ec210c63b3c269f))
* Follow up metrics fixes vol.2
([#656](https://github.com/matter-labs/zksync-era/issues/656))
([5c1aea2](https://github.com/matter-labs/zksync-era/commit/5c1aea2a94d7eded26c3a4ae4973ff983c15e7fa))
* **job-processor:** `max_attepts_reached` metric
([#626](https://github.com/matter-labs/zksync-era/issues/626))
([dd9b308](https://github.com/matter-labs/zksync-era/commit/dd9b308be9b0a6e37aad75f6f54b98e30a2ae14e))
* update google cloud dependencies that do not depend on rsa
([#622](https://github.com/matter-labs/zksync-era/issues/622))
([8a8cad6](https://github.com/matter-labs/zksync-era/commit/8a8cad6ce62f2d34bb34adcd956f6920c08f94b8))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

* ci: removes docker-compose-runner.yml (#649)

## What ❔

<!-- What are the changes this PR brings about? -->
<!-- Example: This PR adds a PR template to the repo. -->
<!-- (For bigger PRs adding more context is appreciated) -->

## Why ❔

<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

* chore(en): adds version metric collecting (#655)

## What ❔

This PR adds a metric that logs release version and protocol version of
the external node. That is needed for provisioning purposes.



## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

* ci(local-node): replace deprecated node setup (#642)

## What ❔

As in commit 33174aa59 and `docker/zk-environment/Dockerfile` use the
repo to setup node, instead of the `curl | sh`.

<!-- What are the changes this PR brings about? -->
<!-- Example: This PR adds a PR template to the repo. -->
<!-- (For bigger PRs adding more context is appreciated) -->

## Why ❔
The old method is deprecated and it saves one minute waiting time
instead of the deprecation banner being displayed.

<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zk fmt` and `zk lint`.
- [ ] Spellcheck has been run via `cargo spellcheck
--cfg=./spellcheck/era.cfg --code 1`.

Signed-off-by: Harald Hoyer <[email protected]>

* chore(en): Add sepolia en config (#663)

## What ❔

- marks goerli testnet EN config as deprecated
- adds sepolia testnet EN config
- removes `EN_BOOTLOADER_HASH`, `EN_DEFAULT_AA_HASH` from configs as
these vars are not used anymore

## Why ❔

Prepare docs for sepolia testnet EN launch

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comme…
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants