Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[16.0][PORT] queu_job: identity_key enhancements (Port from 14.0) #619

Closed
wants to merge 2 commits into from
Closed

[16.0][PORT] queu_job: identity_key enhancements (Port from 14.0) #619

wants to merge 2 commits into from

Commits on Jan 22, 2024

  1. [IMP] queue_job: identity_key enhancements 

    1. In production, a job which is waiting dependencies or which
has started, but not completed, should not be repeated if
the identity_key matches.

2. In tests, the mock queue handler is now enhanced to allow
better mimicking of the identity_key blocks from production.

3. In tests, the mock queue handler now clears the enqueued jobs
after performing them, to better reproduce what a production
environment would do.
    Richard deMeester authored and duongtq committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    de39d41 View commit details
    Browse the repository at this point in the history

  2. queue_job: job_record_with_same_identity_key ignore STARTED 

    Good explanation from Guewen:

Depending on the use case, we should or should not include.
As part of my current work, I use Sidekiq daily.
They have a similar feature, but you can set a per-job parameter unique_until with options:
start (that would mean up to ENQUEUED here) or success (that would be up to STARTED here).

Think about this use case: a job refreshes a cache. Data have changed, we create a pending job. Data change again, no new job because the job is still pending.
Job starts. Data change while the job is running. In this very case, we'd like to enqueue a new job otherwise the cache will be outdated.

I reckon that both cases are valid, but I fear adding this state in the domain may, silently and in subtle ways, existing behaviors.
    @simahawk
    simahawk authored and duongtq committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    7fcc575 View commit details
    Browse the repository at this point in the history