Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DNM] Add a zombie test for PVF preparation #5031

Closed

Conversation

AndreiEres
Copy link
Contributor

No description provided.

muharem and others added 30 commits June 26, 2024 08:13
Introduce migration type to remove data associated with a specific
storage of a pallet.

Based on existing `RemovePallet` migration type.

Required for #3820

---------

Co-authored-by: Liam Aharon <[email protected]>
Co-authored-by: Kian Paimani <[email protected]>
This is PR updates jsonrpsee v0.23 which mainly changes:
- Add `Extensions` which we now is using to get the connection id (used
by the rpc spec v2 impl)
- Update hyper to v1.0, http v1.0, soketto and related crates
(hyper::service::make_service_fn is removed)
- The subscription API for the client is modified to know why a
subscription was closed.

Full changelog here:
https://github.com/paritytech/jsonrpsee/releases/tag/v0.23.0

---------

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Closes: #4298

This PR also merges `xcm-fee-payment-runtime-api` module to the
`xcm-runtime-api`.


## TODO

- [x] rename `convert` to `convert_location` and add new one
`convert_account` (opposite direction)
- [x] add to the all testnet runtimes
- [x] check polkadot-js if supports that automatically or if needs to be
added manually polkadot-js/api#5917
- [ ] backport/patch for fellows and release (asap)

## Open questions
- [x] should we merge `xcm-runtime-api` and
`xcm-fee-payment-runtime-api` to the one module `xcm-runtime-api` ?

## Usage
Input:
 - `location:  VersionedLocation`
 
Output:
 - account_id bytes


![image](https://github.com/paritytech/polkadot-sdk/assets/8159517/4607b15a-77d2-462b-806c-606107776c0d)

---------

Co-authored-by: Bastian Köcher <[email protected]>
Add a `promote_fast` extrinsic to the `core-fellowship` pallet to allow
promotions that ignore the promotion cooldown. It comes with a new
`FastPromoteOrigin`.

Supersedes #4778

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: joe petrowski <[email protected]>
Co-authored-by: command-bot <>
This PR upgrades `litep2p` to the latest version and includes the two
fixes:

1. Enables incoming DHT record validation with `litep2p` network
backend.
2. Sets `TCP_NODELAY` flag on TCP & WS sockets in `litep2p` backend, as
it is currently done in `libp2p` backend.

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Enables the `request_revenue` and `notify_revenue` parts of [RFC 5 -
Coretime
Interface](https://polkadot-fellows.github.io/RFCs/approved/0005-coretime-interface.html)

TODO:
- [x] Finish first pass at implementation
- [x] ~~Need to explicitly burn uncollected and dropped revenue~~
Accumulate it instead
- [x] Confirm working on zombienet
- [x] Tests 
- [ ] Enable XCM `request_revenue` sending on Coretime chain on Kusama
and Polkadot

Fixes: #2209

---------

Co-authored-by: Dmitry Sinyavin <[email protected]>
Co-authored-by: command-bot <>
Co-authored-by: s0me0ne-unkn0wn <[email protected]>
Co-authored-by: Dónal Murray <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Introduce an optional auto-increment setup for the IDs of new assets.

---------

Co-authored-by: joe petrowski <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Ensure that the key ownership proof doesn't contain duplicate or
unneeded nodes.

We already have these checks for the bridge messages proof. Just making
them more generic and performing them also for the key ownership proof.

---------

Co-authored-by: Adrian Catangiu <[email protected]>
Partly fixes
#4890 (comment)

Still the offchain API needs to be updated to hyper v1.0 and I opened an
issue for it, it's using low-level http body features that have been
removed
Adding `Runtime::OmniNode` variant + small changes

---------

Co-authored-by: kianenigma <[email protected]>
Update the instructions to work with the latest parachain template on
Polkadot SDK

---------

Co-authored-by: kianenigma <[email protected]>
Co-authored-by: Kian Paimani <[email protected]>
…o Asset Hubs (#4888)

On Westend Asset Hub, we allow Rococo Asset Hub to act as reserve for
any asset native to the Rococo or Ethereum ecosystems (practically
providing Westend access to Ethereum assets through double bridging:
W<>R<>Eth).

On Rococo Asset Hub, we allow Westend Asset Hub to act as reserve for
any asset native to the Westend ecosystem. We also allow Ethereum
contracts to act as reserves for the foreign assets identified by the
same respective contracts locations.

- [x] add emulated tests for various assets (native, trust-based,
foreign/bridged) going AHR -> AHW,
- [x] add equivalent tests for the other direction AHW -> AHR.

This PR is a prerequisite to doing the same for Polkadot<>Kusama bridge.
… Westend) (#4738)

This adds the new `SignedExtension` to Coretime Rococo and Coretime
Westend runtimes.

---------

Co-authored-by: Dónal Murray <[email protected]>
Counterpart of: #4031

cc @paritytech/networking

---------

Signed-off-by: Alexandru Vasile <[email protected]>
Co-authored-by: Sebastian Kunert <[email protected]>
As per #3326 , removes pallet::getter macro usage from
pallet-membership. The syntax StorageItem::<T, I>::get() should be used
instead. Also converts some syntax to turbo and reimplements the removed
getters, following #223

cc @muraca

---------

Co-authored-by: Dónal Murray <[email protected]>
Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Added Instructions for pallet name customisation in the ReadMe

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
CI required markdown step seems to start failing after
#4806

Signed-off-by: Alexandru Gheorghe <[email protected]>
… AURA (#4932)

Recently thought about the special handling we have for asset-hub
chains. They started with relay chain consensus and transitioned to AURA
at some point. However, nobody should be authoring with relay chain
consensus on these chains anymore, the transition is long done.

I propose to remove this special handling, allowing us to unify one more
execution path.
This PR exposes the `RandomKademliaStarted` event from the litep2p
network backend, and then increments the appropriate metrics.

This is part of: #4681.
However, it is more of an effort to debug low peer count 

### Testing Done
- Started a node and fetched queries:
`substrate_sub_libp2p_kademlia_random_queries_total` produces results
for litep2p backend

cc @paritytech/networking

---------

Signed-off-by: Alexandru Vasile <[email protected]>
…set hub (#4870)

- Send bridged WNDs: Penpal Rococo -> AH Rococo -> AH Westend
- Send bridged ROCs: Penpal Westend -> AH Westend -> AH Rococo

The tests send both ROCs and WNDs, for each direction the native asset
is only used to pay for the transport fees on the local AssetHub, and
are not sent over the bridge.

Including the native asset won't be necessary anymore once we get #4375.

---------

Signed-off-by: Adrian Catangiu <[email protected]>
Co-authored-by: command-bot <>
With random connectivity and latency is hard to actually figure it out a
delta in the benchmarking, so disable them in order to get full
deterministic behaviour when measuring performance.

At least on my machine with this configuration the results for
approval-throughput are really similar between subsequent runs:

```
CPU usage, seconds                     total   per block

approval-distribution                36.9025      3.6902
approval-distribution                36.7579      3.6758
approval-distribution                37.0418      3.7042
approval-distribution                37.0339      3.7034
approval-distribution                36.9342      3.6934
approval-distribution                36.7177       3.6718



approval-voting                      52.7756      5.2776
approval-voting                      52.5999      5.2600
approval-voting                      53.2158      5.3216
approval-voting                      53.2493      5.3249
approval-voting                      52.8524      5.2852
approval-voting                      52.8611      5.2861
approval-voting                      52.8210      5.2821
```

---------

Signed-off-by: Alexandru Gheorghe <[email protected]>
This PR fixes the unused warnings in `frame-support-procedural` crate,
raised by the latest stable rust release.
Related to #4523

Extracting part of #1903
(credits to @Lederstrumpf for the high-level strategy), but also
introducing significant adjustments both to the approach and to the
code. The main adjustment is the fact that the `ForkVotingProof` accepts
only one vote, compared to the original version which accepted a
`vec![]`. With this approach more calls are needed in order to report
multiple equivocated votes on the same commit, but it simplifies a lot
the checking logic. We can add support for reporting multiple signatures
at once in the future.

There are 2 things that are missing in order to consider this issue
done, but I would propose to do them in a separate PR since this one is
already pretty big:
- benchmarks/computing a weight for the new extrinsic (this wasn't
present in #1903 either)
- exposing an API for generating the ancestry proof. I'm not sure if we
should do this in the Mmr pallet or in the Beefy pallet

Co-authored-by: Robert Hambrock <[email protected]>

---------

Co-authored-by: Adrian Catangiu <[email protected]>
Bullrich and others added 18 commits July 11, 2024 13:02
Fixed the mentioned issue:
paritytech/command-bot#113 (comment)

Now it will properly comment when the old bot gets triggered.
Following PR for #4941
that removes usage of `sp-std` on templates

`sp-std` crate was proposed to deprecate on
#2101

@kianenigma

---------

Co-authored-by: Kian Paimani <[email protected]>
This improves logging in the xcm-executor to have better debugability
when executing a XCM message.
…cations (#4935)

## Summary

This PR contains migrated code from the Bridges V2
[branch](#4427) from the
old `parity-bridges-common`
[repo](https://github.com/paritytech/parity-bridges-common/tree/bridges-v2).
Even though the PR looks large, it does not (or should not) contain any
significant changes (also not relevant for audit).
This PR is a requirement for permissionless lanes, as they were
implemented on top of these changes.

## TODO

- [x] generate fresh weights for BridgeHubs
- [x] run `polkadot-fellows` bridges zombienet tests with actual runtime
1.2.5. or 1.2.6 to check compatibility
- ☑️ working, checked with 1.2.8 fellows BridgeHubs
- [x] run `polkadot-sdk` bridges zombienet tests
  - ☑️ with old relayer in CI (1.6.5) 
- [x] run `polkadot-sdk` bridges zombienet tests (locally) - with the
relayer based on this branch -
paritytech/parity-bridges-common#3022
- [x] check/fix relayer companion in bridges repo -
paritytech/parity-bridges-common#3022
- [x] extract pruning stuff to separate PR
#4944

Relates to:
paritytech/parity-bridges-common#2976
Relates to:
paritytech/parity-bridges-common#2451

---------

Signed-off-by: Branislav Kontur <[email protected]>
Co-authored-by: Serban Iorga <[email protected]>
Co-authored-by: Svyatoslav Nikolsky <[email protected]>
Co-authored-by: command-bot <>
Added the command sync. Follow up on #4701 

This PR is blocked until a new machine for the sync gets deployed.
Part of: #239

Polkadot address: 12GyGD3QhT4i2JJpNzvMf96sxxBLWymz4RdGCxRH5Rj5agKW

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
A malicious peer can submit random bytes on transaction protocol.
In this case, the peer is not disconnected or reported back to the
peerstore.

This PR ensures the peer's reputation is properly reported.

Discovered during testing:
- #4977


cc @paritytech/networking

Signed-off-by: Alexandru Vasile <[email protected]>
It says `Will be removed after July 2023` but that's not true 😃

---------

Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
This should remove nearly all usage of `sp-std` except:
- bridge and bridge-hubs
- a few of frames re-export `sp-std`, keep them for now
- there is a usage of `sp_std::Writer`, I don't have an idea how to move
it

Please review proc-macro carefully. I'm not sure I'm doing it the right
way.

Note: need `/bot fmt`

---------

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <[email protected]>
The `DhtEvent::ValuePut` was not propagated back to the higher levels.

This PR ensures we'll send the ValuePut event similarly to
`DhtEvent::ValuePutFailed`

### Next Steps
- [ ] A bit more testing

Thanks @alexggh for catching this 🙏 

cc @paritytech/networking

Signed-off-by: Alexandru Vasile <[email protected]>
This will ensure that malicious code can not access other parts of the
project.

Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
As per #3326, removes usage of the `pallet::getter` macro from the
`session` pallet. The syntax `StorageItem::<T, I>::get()` should be used
instead.

Also, adds public functions for compatibility.

NOTE: The `./historical` directory has not been modified.

cc @muraca

polkadot address: 5GsLutpKjbzsbTphebs9Uy4YK6gTN47MAaz6njPktidjR5cp

---------

Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
This is part of the work to further optimize the approval subsystems, if
you want to understand the full context start with reading
#4849 (comment),
however that's not necessary, as this change is self-contained and nodes
would benefit from it regardless of subsequent changes landing or not.

While testing with 1000 validators I found out that the logic for
determining the validators an assignment should be gossiped to is taking
a lot of time, because it always iterated through all the peers, to
determine which are X and Y neighbours and to which we should randomly
gossip(4 samples).

This could be actually optimised, so we don't have to iterate through
all peers for each new assignment, by fetching the list of X and Y peer
ids from the topology first and then stopping the loop once we took the
4 random samples.

With this improvements we reduce the total CPU time spent in
approval-distribution with 15% on networks with 500 validators and 20%
on networks with 1000 validators.

## Test coverage:

`propagates_assignments_along_unshared_dimension` and
`propagates_locally_generated_assignment_to_both_dimensions` cover
already logic and they passed, confirm that there is no breaking change.

Additionally, the approval voting benchmark measure the traffic sent to
other peers, so I confirmed that for various network size there is no
difference in the size of the traffic sent to other peers.

---------

Signed-off-by: Alexandru Gheorghe <[email protected]>
…t functions (#4566)

Introduce transient storage, which behaves identically to regular
storage but is kept only in memory and discarded after every
transaction. This functionality is similar to the `TSTORE` and `TLOAD`
operations used in Ethereum.

The following new host functions have been introduced:
`get_transient_storage`
`set_transient_storage`
`take_transient_storage`
`clear_transient_storage`
`contains_transient_storage`
Note: These functions are declared as `unstable` and thus are not
activated.

---------

Co-authored-by: command-bot <>
Co-authored-by: PG Herveou <[email protected]>
Co-authored-by: Alexander Theißen <[email protected]>
@AndreiEres AndreiEres changed the base branch from master to AndreiEres/prepare-pvf July 16, 2024 13:08
@AndreiEres AndreiEres requested review from athei, acatangiu, cheme, a team and koute as code owners July 16, 2024 13:08
@paritytech-review-bot paritytech-review-bot bot requested a review from a team July 16, 2024 13:09
@AndreiEres AndreiEres closed this Jul 16, 2024
@AndreiEres AndreiEres deleted the AndreiEres/pvf-prepare-zombie branch July 16, 2024 13:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.