Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Queue leak #157

Open
Skyrain-Young opened this issue Sep 13, 2023 · 2 comments
Open

Queue leak #157

Skyrain-Young opened this issue Sep 13, 2023 · 2 comments

Comments

@Skyrain-Young
Copy link

I set a limit on a queue, but it occasionally takes up more.

# sidekiq.yml
:max_retries: 0
production:
  :concurrency: 8
:queues:
  - default
  - fast_task
  - slow_task
:limits:
  default: 8
  fast_task: 8
  slow_task: 2
# Gemfile.lock
ruby 2.5.1p57
rails (5.1.7)
sidekiq (5.2.10)
  connection_pool (~> 2.2, >= 2.2.2)
  rack (~> 2.0)
  rack-protection (>= 1.5.0)
  redis (~> 4.5, < 4.6.0)
sidekiq-limit_fetch (3.4.0)
  sidekiq (>= 4)

I want the "slow task" queue to execute at most 2 tasks simultaneously. It works fine most of the time, but sometimes it runs more tasks at the same time, like even 7 tasks.

@matt-domsch-sp
Copy link
Contributor

Same here, using latest limit_fetch, sidekiq 6 latest, and AWS Elasticache redis 5.0.6. Looking at the queue explain shows that it has "lost" the locks on several Processes so thinks it should be safe to run additional copies (which it should not). It seems to happen more often under high CPU load on the sidekiq servers.

Repository owner deleted a comment from Skyrain-Young Feb 26, 2024
@deanpcmad
Copy link
Owner

I can't seem to replicate this issue, but if you manage to find it, feel free to send a PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants