Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to install Quickwit with IPv6 network #118

Open
erebe opened this issue Dec 31, 2024 · 0 comments
Open

Unable to install Quickwit with IPv6 network #118

erebe opened this issue Dec 31, 2024 · 0 comments

Comments

@erebe
Copy link

erebe commented Dec 31, 2024

Hello,

I am reporting an issue where I can't install Quickwit chart when the kubernetes cluster is in IPv6.
The reason is due to the healthchecks port, not listening for IPv6.

My value file.

❯ cat quickwit.yaml
environment:
  # QW_LISTEN_ADDRESS: "::"
  QW_METASTORE_URI: s3://quickwit/quickwit-indexes

config:
  # listen_address: "::"
  default_index_root_uri: s3://quickwit/quickwit-indexes
  storage:
    s3:
      flavor: minio
      endpoint: http://[fd00:cafe::6]:9001
      region: eu-east-1
      access_key_id: admin
      secret_access_key: xxxx

tolerations:
  - key: "kubernetes.io/hostname"
    operator: "Equal"
    value: "toybox"
    effect: "NoSchedule"

The helm command used to install quickwit

helm install quickwit quickwit/quickwit -f quickwit.yaml --wait --atomic

The result
image

All pods fails on the startup probe

Startup probe failed: Get "http://[fd01::430]:7280/health/livez": dial tcp [fd01::430]:7280: connect: connection refused

When I get into the machine to check the openned port (I use nsenter to get access to the network namespace)

erebe@toybox ~ [SIGINT]> sudo nsenter -n -t (ps aux | grep 'metastore' | grep -v grep | awk '{print $2}') -- ss -lntp
State           Recv-Q          Send-Q                   Local Address:Port                   Peer Address:Port          Process
LISTEN          0               128                            0.0.0.0:7280                        0.0.0.0:*              users:(("quickwit",pid=200495,fd=13))
LISTEN          0               128                            0.0.0.0:7281                        0.0.0.0:*              users:(("quickwit",pid=200495,fd=12))

We see that Quickwit is listening on 0.0.0.0 but not on [::] for IPv6.
We can confirm with curl

# IPv4 works
erebe@toybox ~ [1]> sudo nsenter -n -t (ps aux | grep 'metastore' | grep -v grep | awk '{print $2}') -- curl http://10.42.4.34:7280/health/livez
true⏎  

#IPv6 hangs
sudo nsenter -n -t (ps aux | grep 'metastore' | grep -v grep | awk '{print $2}') -- curl http://[fd01::430]:7280/health/livez
xxxx

Changing the listening address to :: in the config makes everything works and should be working also for both IPv4 and IPv6 networks

erebe@toybox ~ [SIGINT]> sudo nsenter -n -t (ps aux | grep 'metastore' | grep -v grep | awk '{print $2}') -- ss -lntp
State           Recv-Q          Send-Q                   Local Address:Port                   Peer Address:Port          Process
LISTEN          0               128                                  *:7280                              *:*              users:(("quickwit",pid=204513,fd=13))
LISTEN          0               128                                  *:7281                              *:*              users:(("quickwit",pid=204513,fd=12))

Expected result

The chart should use by default for listening address :: instead of 0.0.0.0
One strange thing to note, I don't know if it is linked to the base image of the container, but usually 0.0.0.0 bind also on IPv6 in order to ease transition from IPv4 to IPv6.
In the case of Quickwit, it does not happen and only bind for IPv4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant