tasks: Regularly clean up leftover containers and outdated images #630
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jelly
reviewed
Dec 6, 2024
| # remove leftover containers | ||
| ExecStart=/bin/sh -ec 'podman ps -q --filter status=exited | xargs -r podman rm' | ||
| # remote old task images; this is opportunistic -- it won't remove used images, but will fail on them | ||
| ExecStart=-/usr/bin/podman rmi --all |
There was a problem hiding this comment.
Never seen ExecStart=-, is that allowed to fail syntax?
There was a problem hiding this comment.
Why not use podman image prune? Should be safer and only remove unused images.
There was a problem hiding this comment.
Yes, - prefix means "allowed to fail", see manpage.
podman image prune
TIL! This is nicer indeed, done.
There was a problem hiding this comment.
(--force is terribly named -- should be -y or so..)
jelly
previously approved these changes
Dec 6, 2024
| Type=oneshot | ||
| # remove leftover containers | ||
| ExecStart=/bin/sh -ec 'podman ps -q --filter status=exited | xargs -r podman rm' | ||
| # remote old task images |
Our bots accumulate outdated task containers which eat a lot of space over time. This will become worse now with automated tasks updates from commit 4824188. Our machinery also sometimes leaks exited/failed containers, clean them up as well.
Our webhook has ran in CentOS CI for a long time. tasks/README.md also says so, but the AWS documentation contradicted that. Update it accordingly.
jelly
approved these changes
Dec 6, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Our bots accumulate outdated task containers which eat a lot of space over time. This will become worse now with automated tasks updates from commit 4824188.
Our machinery also sometimes leaks exited/failed containers, clean them up as well.
Our bot instances tend to look like this:
I rolled this out, the timer started, and cleaned up all that cruft.