Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: rpc helper improvements + core-api update #13

Merged
merged 20 commits into from
Oct 3, 2024

Conversation

Seth-Schmidt
Copy link
Collaborator

Fixes #12

Checklist

  • My branch is up-to-date with upstream/develop branch.
  • Everything works and tested for Python 3.8.0 and above.
  • I ran pre-commit checks against my changes.
  • I've written tests against my changes and all the current present tests are passing.

Current behaviour

Rate limiting is currently not supported in RpcHelper

Additionally, core-api is not compatible with the latest architecture changes.

New expected behaviour

RpcHelper now supports rate limiting on a per-second basis using the aiolimiter leaky bucket method. The limits are configurable on a per-node basis as each RPC will have its own limits. A wrapper has been added around any outgoing requests that will check and wait for resources to become available before continuing with the request.

I have also removed all references to the now unused redis_conn argument for RpcHelper calls, and fixed a response parsing issue I found in batch_eth_get_balance_on_block_range.


Core-Api has been updated for the latest architecture changes. I have also updated the get_project_last_finalized_epoch_info endpoint to no longer iterate over all previous epochs until a finalization is found. I have replaced this in favor of the lastFinalizedSnapshot contract call to the Protocol State that is now available.

Change logs

Added

  • snapshotter/tests/rpc_helper_test.py
  • snapshotter/tests/static/abi/UUPSUpgradeable.json
  • snapshotter/tests/static/bytecode/protocol_state.json
  • snapshotter/tests/static/bytecode/uups_upgradeable.json

Changed

  • poetry.lock
  • pyproject.toml
  • setup.cfg
  • snapshotter/processor_distributor.py
  • snapshotter/snapshotter_id_ping.py
  • snapshotter/system_event_detector.py
  • snapshotter/tests/test_web3_async_provider.py
  • snapshotter/utils/generic_worker.py
  • snapshotter/utils/models/settings_model.py
  • snapshotter/utils/preloaders/tx_receipts/delegated_worker/tx_receipts.py
  • snapshotter/utils/rpc.py
  • snapshotter/utils/snapshot_utils.py

Fixed

  • snapshotter/core_api.py
  • build.sh

Deployment Instructions

No Changes

Copy link
Member

@xadahiya xadahiya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty good, let's deploy this on our test instance to see how it works.

@xadahiya xadahiya self-requested a review October 3, 2024 09:43
Copy link
Member

@xadahiya xadahiya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pushed some minor improvements and cleanup in 842054c. LGTM now!

@xadahiya xadahiya merged commit e993fce into main Oct 3, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement rate limiting for RpcHelper
2 participants