Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change shutdown timeout #1602

Merged
merged 1 commit into from
Jul 13, 2022
Merged

Change shutdown timeout #1602

merged 1 commit into from
Jul 13, 2022

Conversation

tesshucom
Copy link
Owner

@tesshucom tesshucom commented Jul 13, 2022

Problem description

Change shutdown timeout to 29 seconds. This is because the timeout phase timeout value, including Docker, is fix.

  • In a general environment, it is often set to about 30 seconds.
  • For this reason, important processing had a grace period of about 30 seconds. (Many threading await times had been set to 30 seconds)
  • This fix will change the entire shutdown phase to 29 seconds (Many threading await times will be set to 25 seconds)

Additional notes

  • In Docker this value is managed by stop_grace_period, which is 10 seconds by default. A light weight shutdown. That's good, but 10 seconds is rough.
  • Jpdonic assumes that stop_grace_period is 30s.
  • The settings related to the number of seconds should be as follows:
    • Docker : stop_grace_period > Spring : spring.lifecycle.timeout-per-shutdown-phase > The await termination value of ThreadPool
    • ExecutorConfiguration manages threads with long processing time. We can protect from a violent shutdown, by setting the appropriate '@dependon' for these Pooling, cache, and 'liquibase' been.

This is the current workaround.

 - In a general environment, it is often set to about 30 seconds.
 - For this reason, important processing had a grace period of about 30 seconds.
 - This fix will change the entire shutdown phase to 29 seconds
@tesshucom tesshucom added type: enhancement A general enhancement in: data Issues in data (jdbc, db, dao, liquibase) in: docker Issues in the test module. in: threading Issues in threading labels Jul 13, 2022
@tesshucom tesshucom self-assigned this Jul 13, 2022
@tesshucom
Copy link
Owner Author

tesshucom commented Jul 13, 2022

The mechanism of Spring's graceful shutdown and the mechanism of exit code are closely related. There are some traps, and the behavior depends on the version of Spring Boot. Therefore, fragmentary changes can cause deadlocks. Best practices do not appear to be finalized.

What is being done with Jpsonic is a workaround that assumes a shutdown hook or a graceful-shutdown and can work with the latest Spring. Spring Boot versions are not stagnant or reverted for a particular implementation. The old Spring Boot implementation method doesn't help. The #839 needs to be resolved in the end, so it should always be checked with the latest version.

https://github.com/spring-projects/spring-boot/issues/27049
https://github.com/spring-projects/spring-boot/issues/23625

Explicit control of the shutdown code is currently difficult. Therefore, Jpsonic will reply with the received signal as it is. (That is 143, for Docker)

If the exit code mapping by exception does not cause a deadlock, there may be a tricky workaround to take advantage of it... In other words, it seems better not to uselessly implement any more. If Spring Boot provides good practices in the future, you should follow them.

Exit code issues are not related to issues such as:

https://hynek.me/articles/docker-signals/

Message transmission from Docker to Java is successful. (We can receive it, but the latest Spring Boot does not allow good coexistence of exit code control and shutdown process.)

Therefore, at present, it is a realistic idea to map the exit code with tini as follows.

ENTRYPOINT ["tini", "-e", "143", "--", "run.sh"]

It may be better to maintain the above method for some time.


  • In SpringBoot, the shutdown hook, context close, and exit code control conflict now. If this is resolved, better implementation may be possible
  • It seems that a standard shutdown implementation is being considered in Liquibase as well. This may also be positively affected in the future

With these in mind, realistic workarounds should be accepted. For now!

@tesshucom tesshucom merged commit b6c7a51 into develop Jul 13, 2022
@tesshucom tesshucom deleted the feature/wait-time4shutdown branch July 13, 2022 17:10
@tesshucom tesshucom added this to the jpsonic 111.3.0 milestone Jul 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
in: data Issues in data (jdbc, db, dao, liquibase) in: docker Issues in the test module. in: threading Issues in threading type: enhancement A general enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant