Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

One of the OVH nodes is out of disk space #2854

Closed
manics opened this issue Dec 6, 2023 · 2 comments
Closed

One of the OVH nodes is out of disk space #2854

manics opened this issue Dec 6, 2023 · 2 comments

Comments

@manics
Copy link
Member

manics commented Dec 6, 2023

$ kubectl describe node builds-2304-node-ff87f2
...
Events:
  Type     Reason               Age                     From     Message
  ----     ------               ----                    ----     -------
  Warning  FreeDiskSpaceFailed  2m46s (x3041 over 11d)  kubelet  failed to garbage collect required amount of images. Wanted to free 41585983488 bytes, but freed 0 bytes

Is it safe to run kubectl delete node builds-2304-node-ff87f2 to force the node to be replaced?

@minrk
Copy link
Member

minrk commented Dec 7, 2023

Yes, if nothing healthy is on it. Or you can cordon and drain and wait if there's a user build you want to let finish (unlikely if disk is full)

@manics
Copy link
Member Author

manics commented Dec 7, 2023

Thanks! The node no longer exists, so either it got scaled down automatically or someone else deleted it.

@manics manics closed this as completed Dec 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants