Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Abort async tasks using asyncio #1190

Merged
merged 5 commits into from
Sep 12, 2024

Conversation

onlyann
Copy link
Contributor

@onlyann onlyann commented Sep 8, 2024

Closes #1084

Successful PR Checklist:

  • Tests
    • (not applicable?)
  • Documentation
    • (not applicable?)

PR label(s):

Changes

This modifies the way jobs are being aborted when the task is async, by cancelling the underlying asyncio Task that represents the running job.

@onlyann onlyann requested a review from a team as a code owner September 8, 2024 11:22
@github-actions github-actions bot added the PR type: feature ⭐️ Contains new features label Sep 8, 2024
@onlyann onlyann changed the title Cancel-async-tasks Abort async tasks Sep 8, 2024
@onlyann onlyann changed the title Abort async tasks Abort async tasks using asyncio Sep 8, 2024
Copy link

github-actions bot commented Sep 8, 2024

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  procrastinate
  worker.py 398
Project Total  

This report was generated by python-coverage-comment-action

Copy link
Member

@ewjoachim ewjoachim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll need to make a more thorough review soon but it seems good :)

await do_something()
```

It is possible to prevent the job from aborting by capturing asyncio.CancelledError.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
It is possible to prevent the job from aborting by capturing asyncio.CancelledError.
It is possible to prevent the job from aborting by [shielding](https://docs.python.org/3/library/asyncio-task.html#shielding-from-cancellation) (and/or with `except asyncio.CancelledError`.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically, what prevents the job from being aborted is catching the error, not shielding.

In practice, it is likely the combination of both that the user wants.

How would you prefer to word it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want to have some custom behavior at cancellation time, use a combination of [shielding](https://docs.python.org/3/library/asyncio-task.html#shielding-from-cancellation) and `except asyncio.CancelledError`.

)

for task in tasks_to_cancel:
task.cancel()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's worth logging something (in debug) for each cancelled task. I think I would remove the

            and context.task
            and asyncio.iscoroutinefunction(context.task.func)

above, and instead make a dedicated function that receives a task & context and logs and cancels it if it's a coroutine.

@ewjoachim
Copy link
Member

Trigger CI

Seems like you know how to make an empty commit to trigger CI. Note; you can combine this with git commit --amend to do that on your last commit and not even add an empty commut to trigger the CI again.

@ewjoachim
Copy link
Member

Congratulations again :)

@ewjoachim ewjoachim merged commit 7cc1d00 into procrastinate-org:v3 Sep 12, 2024
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
PR type: feature ⭐️ Contains new features
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants