Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

common, agent: switch most timer calls to sequential impl #1065

Merged
merged 2 commits into from
Dec 26, 2024

Conversation

dwerner
Copy link
Contributor

@dwerner dwerner commented Dec 18, 2024

This PR adds two new functions which implement a sequential timer with back-pressure, as well as using those in place of where we previously used a timer(...).tryMap|pipe|map|reduce.

A long standing issue with timer() is that it's blind to the work done further down the combinator pipeline. So if the work in a timer took longer than the next setInterval backing the timer to complete, we could potentially start executing the next iteration before completing the one we are currently on.

These newly introduced functions watch the amount of work done in their Mapper or Reducer functions respectively.

I'm considering this a stop-gap solution as it provides a single logged function which still makes use of eventuals.

@dwerner dwerner requested review from fordN and gusinacio December 18, 2024 22:39
Copy link
Member

@gusinacio gusinacio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Great work, this will make indexer-agent way more stable.

@dwerner dwerner force-pushed the sequential-timer branch 2 times, most recently from 23cee85 to 4362b8f Compare December 26, 2024 19:00
@dwerner dwerner merged commit 3c4d559 into main Dec 26, 2024
9 checks passed
@dwerner dwerner deleted the sequential-timer branch December 26, 2024 19:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: 🚗 Merged
Development

Successfully merging this pull request may close these issues.

2 participants