Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Worker: Method for selecting task to work on #16

Open
dwhswenson opened this issue May 31, 2023 · 0 comments
Open

Worker: Method for selecting task to work on #16

dwhswenson opened this issue May 31, 2023 · 0 comments

Comments

@dwhswenson
Copy link
Member

The Worker need to have a way to select which task it will run. There are a few options here; I think we should engineer things such that we can easily try alternatives, since I'm not sure what will best meet needs of users. A couple options:

  • Priority in the task status DB. Get first available sorted by priority (on the SQL end). Should be fast, but reading doesn't block other readers, so there are potential concurrency pileup issues.
  • Make the decision in Python; could include some randomness to avoid concurrency issues (e.g., select weighted by priority). Will be slower between read and claim, but we already have safety on the claim to ensure that we're actually the only one to get stake our claim to a task.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant