Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automate requeue of inactive jobs #5

Merged
merged 1 commit into from
Feb 10, 2025
Merged

Conversation

PeterC10
Copy link
Collaborator

@PeterC10 PeterC10 commented Feb 4, 2025

Backport of enhancement from Odoo 16 PR on OCA: OCA#716

Currently testing on Staging.

Need to decide the following:

  • Do we want to requeue the job (as long as max retries not exceeded) or mark it as failed immediately?
    ** A: Use the retries
  • Check how this is reported via Sentry compared to other job errors
    ** A: No error unless logging an error. Instead the requeue_dead_jobs method will return info on the pending/failed jobs, this info can be used to manually report the event to Sentry with the desired tags.
  • If restarting the Odoo instance, any jobs in started state will be automatically started again - need to make sure this isn't likely to cause an issue
    ** A: No issue identified, allows Odoo to get itself back into a healthy state if service is restarted abruptly
  • Whether to keep the current stuck jobs cron job
    ** A: Will keep it, for cases where jobs are running for a long time but have not been killed

@PeterC10 PeterC10 force-pushed the 11.0_DM-583_requeue_dead_jobs branch 2 times, most recently from 3e5f22d to 776c454 Compare February 10, 2025 10:53
@PeterC10 PeterC10 changed the title WIP - Requeue jobs that are not actively running (i.e. CPU timeout) Automate requeue of inactive jobs Feb 10, 2025
@PeterC10 PeterC10 force-pushed the 11.0_DM-583_requeue_dead_jobs branch from 776c454 to 8629acd Compare February 10, 2025 10:54
* Job Queue Lock model added to track jobs being actively processed by Odoo
* Any job not being actively worked on will either be:
    * Requeued, if max_retries not reached
    * Marked as failed, if max_retries reached
* Covers cases where the Odoo instance is restarted or a job is killed by
  Odoo due to exceeding the limit_time_cpu setting

Backport of enhancement from Odoo 16 PR on OCA:  OCA#716
@PeterC10 PeterC10 force-pushed the 11.0_DM-583_requeue_dead_jobs branch from 8629acd to cb47865 Compare February 10, 2025 10:57
@PeterC10 PeterC10 merged commit cfb5ff2 into 11.0 Feb 10, 2025
@PeterC10 PeterC10 deleted the 11.0_DM-583_requeue_dead_jobs branch February 10, 2025 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants