Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

zarf init fails on multi-node minikube cluster #1998

Closed
vanakema opened this issue Aug 30, 2023 · 5 comments · May be fixed by #3241
Closed

zarf init fails on multi-node minikube cluster #1998

vanakema opened this issue Aug 30, 2023 · 5 comments · May be fixed by #3241
Labels
possible-bug 🐛

Comments

@vanakema
Copy link

vanakema commented Aug 30, 2023 •
edited
Loading

Environment

Device and OS: MacBook Pro M2 Pro, MacOS
App version: v0.29.1
Kubernetes distro being used: MiniKube (Docker driver) v1.31.1, k8s v1.24.7
Other:

Steps to reproduce

  1. minikube start --kubernetes-version 1.24.7 --nodes 3
  2. zarf init --confirm

Expected result

It works

Actual Result

The registry fails to deploy

Pod Logs (zarf-docker-registry):

Configuration error: error parsing /etc/docker/registry/config.yml: no storage configuration provided
Usage: 
  registry serve <config> [flags]
Flags:
  -h, --help=false: help for serve
Additional help topics:

Visual Proof (screenshots, videos, text, etc)

It just never deploys, and the Helm deploy just keeps trying till it times out, spitting out essentially the same error as above.

Severity/Priority

High: needed for our developer environment for testing HA application deployments. Not affecting prod (yet)

Additional Information

We tried this as well, which resulted in 1 registry pod deploying, but the 2nd pod had the same error as listed above
zarf init --storage-class=standard --set REGISTRY_PVC_ENABLED=false --confirm
@vanakema
Copy link
Author

This go around of tearing down minikube and bringing it up, it did successfully deploy the registry pod, but the rest of the zarf deploy fails with this error:

     ERROR:  unable to deploy all components in this Zarf Package: unable to deploy component zarf-registry: unable to deploy component zarf-registry: unable to push images to the registry: POST http://127.0.0.1:59435/v2/library/registry/blobs/uploads/: UNKNOWN: unknown error; map[DriverName:filesystem Enclosed:map[Err:13 Op:mkdir Path:/var/lib/registry/docker]]

@Noxsios
Copy link
Contributor

Noxsios commented Sep 3, 2023

@vanakema

Following: https://minikube.sigs.k8s.io/docs/tutorials/volume_snapshots_and_csi/ w/ a fix as noted in kubernetes/minikube#12360

This worked for me:

minikube start --kubernetes-version 1.24.7 --nodes 3
minikube addons enable volumesnapshots
minikube addons enable csi-hostpath-driver
kubectl patch storageclass csi-hostpath-sc -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

zarf init --storage-class=csi-hostpath-sc --log-level=trace --confirm

Let me know if this works for you!

@Noxsios Noxsios mentioned this issue Sep 4, 2023
Closed
@vanakema
Copy link
Author

vanakema commented Sep 7, 2023

Thanks! We went in a different direction where we don't need a cluster locally anymore, but if we do, I'll try it out

@Racer159
Copy link
Contributor

Racer159 commented Dec 6, 2023

Going to close this issue for now - Zarf does require a storage class for its registry by default (though you can turn this off and back it by something like S3 if you prefer) - also worth noting that we are working on a zarf-init-longhorn package that should be ready soon that will include a storage class of its own.

https://github.com/defenseunicorns/zarf-init-longhorn

@Racer159 Racer159 closed this as completed Dec 6, 2023
@Racer159
Copy link
Contributor

Racer159 commented Dec 6, 2023

We can reopen if we need to @vanakema

xyzzyz added a commit to xyzzyz/zarf that referenced this issue Nov 15, 2024
@xyzzyz
Actually use emptydir mount when persistence is disabled
d4b2927 
In zarf-dev#2124, an emptydir was mounted when `persistence.enabled: false` is set. However, this is not
enough to make registry work: it crashloops as in issue zarf-dev#1998. This is fixed by setting the env variable
specifying storage config even if persistence is disabled.
xyzzyz added a commit to xyzzyz/zarf that referenced this issue Nov 15, 2024
@xyzzyz
Actually use emptydir mount when persistence is disabled
ef14c1e 
In zarf-dev#2124, an emptydir was mounted when `persistence.enabled: false` is set. However, this is not
enough to make registry work: it crashloops as in issue zarf-dev#1998. This is fixed by setting the env variable
specifying storage config even if persistence is disabled.
@xyzzyz xyzzyz mentioned this issue Nov 15, 2024
Open
2 tasks
xyzzyz added a commit to xyzzyz/zarf that referenced this issue Nov 15, 2024
@xyzzyz
Actually use emptydir mount when persistence is disabled
984a56f 
In zarf-dev#2124, an emptydir was mounted when `persistence.enabled: false` is set. However, this is not
enough to make registry work: it crashloops as in issue zarf-dev#1998. This is fixed by setting the env variable
specifying storage config even if persistence is disabled.

Signed-off-by: Adam Michalik <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
possible-bug 🐛
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Actually use emptydir mount when persistence is disabled xyzzyz/zarf
3 participants
@vanakema @Racer159 @Noxsios