Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test benchmarks #79

Open
wants to merge 16 commits into
base: master
Choose a base branch
from
Open

Test benchmarks #79

wants to merge 16 commits into from

Conversation

iulianbarbu
Copy link

test PR

seemantaggarwal and others added 14 commits January 22, 2025 10:53
…embed it in its README paritytech#6333 (paritytech#7093)

Use docify export for parachain template hardcoded configuration and
embed it in its README paritytech#6333

Docify currently has a limitation of not being able to embed a
variable/const in its code, without embedding it's definition, even if
do something in a string like

"this is a sample string ${sample_variable}"

It will embed the entire string 
"this is a sample string ${sample_variable}"
without replacing the value of sample_variable from the code

Hence, the goal was just to make it obvious in the README where the
PARACHAIN_ID value is coming from, so a note has been added at the start
for the same, so whenever somebody is running these commands, they will
be aware about the value and replace accordingly.

To make it simpler, we added a 
rust ignore block so the user can just look it up in the readme itself
and does not have to scan through the runtime directory for the value.

---------

Co-authored-by: Iulian Barbu <[email protected]>
…aritytech#7253)

Saw this test flake a few times, last time
[here](https://github.com/paritytech/polkadot-sdk/actions/runs/12834432188/job/35791830215).

We first fetch all processes in the test, then query `/proc/<pid>/stat`
for every one of them. When the file was not found, we would error. Now
we tolerate not finding this file. Ran 200 times locally without error,
before would fail a few times, probably depending on process fluctuation
(which I expect to be high on CI runners).
Log error instead of failing with an error when block processing fails

---------

Co-authored-by: command-bot <>
…ghts. (paritytech#7098)

This PR addresses a few minor issues found while working on the
polkadot-fellows PR
[https://github.com/polkadot-fellows/runtimes/pull/522](https://github.com/polkadot-fellows/runtimes/pull/522):
- Incorrect generic type for `InboundLaneData` in
`check_message_lane_weights`.
- Renaming leftovers: `assigner_on_demand` -> `on_demand`.
Part of paritytech#6504

### Added modules

- `utility`: Traits not tied to any direct operation in the runtime.

polkadot address: 14SRqZTC1d8rfxL8W1tBTnfUBPU23ACFVPzp61FyGf4ftUFg

---------

Co-authored-by: Giuseppe Re <[email protected]>
…h#7263)

## Description

The `frame-benchmarking-cli` crate has not been buildable without the
`rocksdb` feature since version 1.17.0.

**Error:**  
```rust
self.database()?.unwrap_or(Database::RocksDb),
                             ^^^^^^^ variant or associated item not found in `Database`
```

This issue is also related to the `rocksdb` feature bleeding (paritytech#3793),
where the `rocksdb` feature was always activated even when compiling
this crate with `--no-default-features`.

**Fix:**  
- Resolved the error by choosing `paritydb` as the default database when
compiled without the `rocksdb` feature.
- Fixed the issue where the `sc-cli` crate's `rocksdb` feature was
always active, even compiling `frame-benchmarking-cli` with
`--no-default-features`.

## Review Notes

Fix the crate to be built without rocksdb, not intended to solve paritytech#3793.

---------

Co-authored-by: command-bot <>
…ritytech#7260)

For eth-indexer, it's probably safer to use `subscribe_finalized` and
index these blocks into the DB rather than `subscribe_best`

---------

Co-authored-by: command-bot <>
Currently `pallet-collator-selection` does not expose a way to query the
assigned pot account derived from the `PotId` configuration item.
Without it, it is not possible to transfer the existential deposit to
it.

This PR addresses this issue by exposing an extra constant.

---------

Co-authored-by: Bastian Köcher <[email protected]>
Copies all the integration tests from Rococo to Westend.

Closes: paritytech#6389
Link-checker job is constantly failing because of these two links.

In the browser there is a redirect, apparently our lychee checker can't
handle it.
Part of  paritytech#6504

---------

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Giuseppe Re <[email protected]>
Signed-off-by: Iulian Barbu <[email protected]>
@cla-bot-2021-stg
Copy link

User @iulianbarbu, please sign the CLA here.

@iulianbarbu
Copy link
Author

/cmd bench --runtime westend --pallet pallets_balances

Copy link

Command "bench --runtime westend --pallet pallets_balances" has started 🚀 See logs here

Copy link

Command "bench --runtime westend --pallet pallets_balances" has failed ❌! See logs here

@iulianbarbu
Copy link
Author

/cmd bench --runtime westend --pallet pallet_balances

Copy link

Command "bench --runtime westend --pallet pallet_balances" has started 🚀 See logs here

Copy link

Command "bench --runtime westend --pallet pallet_balances" has failed ❌! See logs here

2 similar comments
Copy link

Command "bench --runtime westend --pallet pallet_balances" has failed ❌! See logs here

Copy link

Command "bench --runtime westend --pallet pallet_balances" has failed ❌! See logs here

@iulianbarbu
Copy link
Author

/cmd bench --runtime westend --pallet pallet_balances

Copy link

Command "bench --runtime westend --pallet pallet_balances" has started 🚀 See logs here

Copy link

Command "bench --runtime westend --pallet pallet_balances" has finished ✅ See logs here

Subweight results:
File Extrinsic Old New Change [%]
cumulus/pallets/collator-selection/src/weights.rs leave_intent - - ERROR
cumulus/pallets/collator-selection/src/weights.rs new_session - - ERROR
cumulus/pallets/collator-selection/src/weights.rs register_as_candidate - - ERROR
cumulus/pallets/collator-selection/src/weights.rs set_invulnerables - - ERROR
cumulus/pallets/collator-selection/src/weights.rs take_candidate_slot - - ERROR
cumulus/pallets/collator-selection/src/weights.rs update_bond - - ERROR
cumulus/parachains/runtimes/assets/asset-hub-rococo/src/weights/pallet_collator_selection.rs take_candidate_slot - - ERROR
cumulus/parachains/runtimes/assets/asset-hub-rococo/src/weights/pallet_collator_selection.rs update_bond - - ERROR
cumulus/parachains/runtimes/assets/asset-hub-westend/src/weights/pallet_collator_selection.rs take_candidate_slot - - ERROR
cumulus/parachains/runtimes/assets/asset-hub-westend/src/weights/pallet_collator_selection.rs update_bond - - ERROR
cumulus/parachains/runtimes/bridge-hubs/bridge-hub-rococo/src/weights/pallet_collator_selection.rs take_candidate_slot - - ERROR
cumulus/parachains/runtimes/bridge-hubs/bridge-hub-rococo/src/weights/pallet_collator_selection.rs update_bond - - ERROR
cumulus/parachains/runtimes/bridge-hubs/bridge-hub-westend/src/weights/pallet_collator_selection.rs take_candidate_slot - - ERROR
cumulus/parachains/runtimes/bridge-hubs/bridge-hub-westend/src/weights/pallet_collator_selection.rs update_bond - - ERROR
cumulus/parachains/runtimes/collectives/collectives-westend/src/weights/pallet_collator_selection.rs take_candidate_slot - - ERROR
cumulus/parachains/runtimes/collectives/collectives-westend/src/weights/pallet_collator_selection.rs update_bond - - ERROR
cumulus/parachains/runtimes/collectives/collectives-westend/src/weights/pallet_preimage.rs ensure_updated - - ERROR
cumulus/parachains/runtimes/people/people-rococo/src/weights/pallet_collator_selection.rs take_candidate_slot - - ERROR
cumulus/parachains/runtimes/people/people-rococo/src/weights/pallet_collator_selection.rs update_bond - - ERROR
cumulus/parachains/runtimes/people/people-westend/src/weights/pallet_collator_selection.rs take_candidate_slot - - ERROR
cumulus/parachains/runtimes/people/people-westend/src/weights/pallet_collator_selection.rs update_bond - - ERROR
polkadot/runtime/westend/src/weights/pallet_preimage.rs ensure_updated - - ERROR
substrate/frame/election-provider-support/src/weights.rs phragmen - - ERROR
substrate/frame/election-provider-support/src/weights.rs phragmms - - ERROR
substrate/frame/beefy-mmr/src/weights.rs read_peak 31.76us 35.80us +12.70
polkadot/runtime/westend/src/weights/pallet_beefy_mmr.rs n_items_proof_is_non_canonical 786.19us 829.74us +5.54
polkadot/runtime/westend/src/weights/pallet_beefy_mmr.rs extract_validation_context 35.33us 33.17us -6.11
substrate/frame/beefy-mmr/src/weights.rs n_items_proof_is_non_canonical 805.03us 576.24us -28.42
Command output:

✅ Successful benchmarks of runtimes/pallets:
-- westend: ['pallet_balances']

@cla-bot-2021-stg
Copy link

User @seemantaggarwal, please sign the CLA here.

User @skunert, please sign the CLA here.

User @pgherveou, please sign the CLA here.

User @bkontur, please sign the CLA here.

User @yrong, please sign the CLA here.

User @runcomet, please sign the CLA here.

User @re-gius, please sign the CLA here.

User @bgallois, please sign the CLA here.

User @Moliholy, please sign the CLA here.

User @claravanstaden, please sign the CLA here.

User @Krayt78, please sign the CLA here.

@iulianbarbu
Copy link
Author

/cmd bench-omni --runtime westend --pallet pallet_balances

Copy link

Command "bench-omni --runtime westend --pallet pallet_balances" has started 🚀 See logs here

Copy link

Command "bench-omni --runtime westend --pallet pallet_balances" has finished ✅ See logs here

Subweight results:
File Extrinsic Old New Change [%]
cumulus/pallets/collator-selection/src/weights.rs leave_intent - - ERROR
cumulus/pallets/collator-selection/src/weights.rs new_session - - ERROR
cumulus/pallets/collator-selection/src/weights.rs register_as_candidate - - ERROR
cumulus/pallets/collator-selection/src/weights.rs set_invulnerables - - ERROR
cumulus/pallets/collator-selection/src/weights.rs take_candidate_slot - - ERROR
cumulus/pallets/collator-selection/src/weights.rs update_bond - - ERROR
cumulus/parachains/runtimes/assets/asset-hub-rococo/src/weights/pallet_collator_selection.rs take_candidate_slot - - ERROR
cumulus/parachains/runtimes/assets/asset-hub-rococo/src/weights/pallet_collator_selection.rs update_bond - - ERROR
cumulus/parachains/runtimes/assets/asset-hub-westend/src/weights/pallet_collator_selection.rs take_candidate_slot - - ERROR
cumulus/parachains/runtimes/assets/asset-hub-westend/src/weights/pallet_collator_selection.rs update_bond - - ERROR
cumulus/parachains/runtimes/bridge-hubs/bridge-hub-rococo/src/weights/pallet_collator_selection.rs take_candidate_slot - - ERROR
cumulus/parachains/runtimes/bridge-hubs/bridge-hub-rococo/src/weights/pallet_collator_selection.rs update_bond - - ERROR
cumulus/parachains/runtimes/bridge-hubs/bridge-hub-westend/src/weights/pallet_collator_selection.rs take_candidate_slot - - ERROR
cumulus/parachains/runtimes/bridge-hubs/bridge-hub-westend/src/weights/pallet_collator_selection.rs update_bond - - ERROR
cumulus/parachains/runtimes/collectives/collectives-westend/src/weights/pallet_collator_selection.rs take_candidate_slot - - ERROR
cumulus/parachains/runtimes/collectives/collectives-westend/src/weights/pallet_collator_selection.rs update_bond - - ERROR
cumulus/parachains/runtimes/collectives/collectives-westend/src/weights/pallet_preimage.rs ensure_updated - - ERROR
cumulus/parachains/runtimes/people/people-rococo/src/weights/pallet_collator_selection.rs take_candidate_slot - - ERROR
cumulus/parachains/runtimes/people/people-rococo/src/weights/pallet_collator_selection.rs update_bond - - ERROR
cumulus/parachains/runtimes/people/people-westend/src/weights/pallet_collator_selection.rs take_candidate_slot - - ERROR
cumulus/parachains/runtimes/people/people-westend/src/weights/pallet_collator_selection.rs update_bond - - ERROR
polkadot/runtime/westend/src/weights/pallet_preimage.rs ensure_updated - - ERROR
substrate/frame/election-provider-support/src/weights.rs phragmen - - ERROR
substrate/frame/election-provider-support/src/weights.rs phragmms - - ERROR
substrate/frame/beefy-mmr/src/weights.rs read_peak 31.76us 35.80us +12.70
polkadot/runtime/westend/src/weights/pallet_beefy_mmr.rs n_items_proof_is_non_canonical 786.19us 829.74us +5.54
polkadot/runtime/westend/src/weights/pallet_beefy_mmr.rs extract_validation_context 35.33us 33.17us -6.11
substrate/frame/beefy-mmr/src/weights.rs n_items_proof_is_non_canonical 805.03us 576.24us -28.42
Command output:

✅ Successful benchmarks of runtimes/pallets:
-- westend: ['pallet_balances']

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.