Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Event handlers that do not finish can block the whole backend #1751

Open
pcdavid opened this issue Feb 22, 2023 · 0 comments · May be fixed by #1752
Open

Event handlers that do not finish can block the whole backend #1751

pcdavid opened this issue Feb 22, 2023 · 0 comments · May be fixed by #1752

Comments

@pcdavid
Copy link
Member

pcdavid commented Feb 22, 2023

In EditingContextEventProcessor, we have:

Future<?> future = this.executorService.submit(() -> this.doHandle(payloadSink, input));
try {
  // Block until the event has been processed
  future.get();
} catch (...) {
  //
}

The future.get() call has no timeout, so if for some reason the handler does not terminate (infinite loop, deadlock...), the http handler thread which executes EditingContextEventProcessor stays blocked indefinitely. If enough http requests trigger the offending handler, we can get to a point where enough threads are holding server resources (e.g. JDBC connections) that new requests can not be handled at all.

We can't do much to prevent buggy event handlers to run forever, but we can probably avoid the blocking to affect the EditingContextEventProcessor's thread.

pcdavid added a commit that referenced this issue Feb 22, 2023
@pcdavid pcdavid linked a pull request Feb 22, 2023 that will close this issue
39 tasks
@sbegaudeau sbegaudeau added this to the 2023.4.0 milestone Feb 23, 2023
@sbegaudeau sbegaudeau linked a pull request Feb 23, 2023 that will close this issue
39 tasks
@sbegaudeau sbegaudeau removed this from the 2023.4.0 milestone Apr 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants