Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature request: speed up data pipeline #960

Open
wakamex opened this issue Oct 10, 2023 · 3 comments
Open

feature request: speed up data pipeline #960

wakamex opened this issue Oct 10, 2023 · 3 comments
Assignees
Labels
enhancement A feature or refactor request

Comments

@wakamex
Copy link
Contributor

wakamex commented Oct 10, 2023

currently, running arbitrary bot actions back-to-back can't be done right away. that's because the second execution instantiates its wallet from the chainsync process running in the background, which may be a few milliseconds behind. I found a time.sleep(1) call to be sufficient to let it catch up. however, this is a bit messy and sub-optimal. Avoiding that would be ideal. A few ways to do so are:

  1. create a wait_for_data() function which waits for chainsync to update to the latest block
  2. allow instantiating bot wallets directly from the chain, without the use of chainsync. as discussed at the offsite, this is the only case where bots require the data backend to run, so creating an alternate method would remove this dependency.
@wakamex wakamex added the enhancement A feature or refactor request label Oct 10, 2023
@slundqui
Copy link
Contributor

This is a known issue. Unfortunately, solution number 2 isn't feasible because anvil loses state after some time, and getting shorts/longs needs deltas from the beginning of time. I'll look into solution 1, but on the server side to ensure the analysis pipeline is up to date before returning

@dpaiton
Copy link
Member

dpaiton commented Oct 10, 2023

Marking this as backlog until after the 17th

@dpaiton dpaiton changed the title feature request: avoid need for delay between bot actions feature request: speed up data pipeline Jun 5, 2024
@dpaiton
Copy link
Member

dpaiton commented Jun 5, 2024

need to explore with new refactors, might be able to run data pipeline async

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement A feature or refactor request
Projects
None yet
Development

No branches or pull requests

3 participants