Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to connect to CoreDNS instances when running local tutorial #1830

Open
mattwelke opened this issue Feb 11, 2025 · 2 comments · May be fixed by #1832
Open

Unable to connect to CoreDNS instances when running local tutorial #1830

mattwelke opened this issue Feb 11, 2025 · 2 comments · May be fixed by #1832

Comments

@mattwelke
Copy link

mattwelke commented Feb 11, 2025 •
edited
Loading

When running the tutorial at https://www.k8gb.io/docs/local.html, it works for me up until the point where you run dig for each instance. I get an error saying it can't connect:

dig -p 5053 +tcp @localhost localtargets-roundrobin.cloud.example.com && \
dig -p 5054 +tcp @localhost localtargets-roundrobin.cloud.example.com
;; communications error to 127.0.0.1#5053: connection reset
;; communications error to 127.0.0.1#5053: connection reset
;; communications error to 127.0.0.1#5053: connection reset

; <<>> DiG 9.18.30-0ubuntu0.24.04.2-Ubuntu <<>> -p 5053 +tcp @localhost localtargets-roundrobin.cloud.example.com
; (1 server found)
;; global options: +cmd
;; no servers could be reached

I am however able to run the dig command slightly above that in the tutorial where I connect to the EdgeDNS cluster:

$ dig @localhost -p 1053 roundrobin.cloud.example.com +short +tcp
172.19.0.8
172.19.0.7
172.19.0.4
172.19.0.5

I figured to help with troubleshooting, I should include info about the services that are setup in each cluster at this point in time:

$ k config use k3d-edgedns
Switched to context "k3d-edgedns".

$ k get svc -A
NAMESPACE     NAME         TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)                     AGE
default       bind         NodePort    10.43.63.77   <none>        53:30053/TCP,53:30053/UDP   3h13m
default       kubernetes   ClusterIP   10.43.0.1     <none>        443/TCP                     3h14m
kube-system   kube-dns     ClusterIP   10.43.0.10    <none>        53/UDP,53/TCP,9153/TCP      3h14m

$ k config use k3d-test-gslb1
Switched to context "k3d-test-gslb1".

$ k get svc -A
NAMESPACE         NAME                   TYPE           CLUSTER-IP      EXTERNAL-IP             PORT(S)                                 AGE
default           kubernetes             ClusterIP      10.43.0.1       <none>                  443/TCP                                 3h15m
istio-ingress     istio-ingressgateway   LoadBalancer   10.43.171.174   172.19.0.4,172.19.0.5   8080:31999/TCP                          3h14m
istio-system      istiod                 ClusterIP      10.43.154.83    <none>                  15010/TCP,15012/TCP,443/TCP,15014/TCP   3h14m
k8gb              k8gb-coredns           ClusterIP      10.43.64.73     <none>                  53/UDP                                  3h15m
kube-system       kube-dns               ClusterIP      10.43.0.10      <none>                  53/UDP,53/TCP,9153/TCP                  3h15m
test-gslb-istio   frontend-podinfo       ClusterIP      10.43.156.0     <none>                  9898/TCP,9999/TCP                       3h14m
test-gslb-istio   unhealthy-app          ClusterIP      10.43.93.229    <none>                  80/TCP                                  3h14m
test-gslb         frontend-podinfo       ClusterIP      10.43.218.93    <none>                  9898/TCP,9999/TCP                       3h14m
test-gslb         unhealthy-app          ClusterIP      10.43.130.87    <none>                  80/TCP                                  3h14m

$ k config use k3d-test-gslb2
Switched to context "k3d-test-gslb2".

$ k get svc -A
NAMESPACE         NAME                   TYPE           CLUSTER-IP      EXTERNAL-IP             PORT(S)                                 AGE
default           kubernetes             ClusterIP      10.43.0.1       <none>                  443/TCP                                 3h16m
istio-ingress     istio-ingressgateway   LoadBalancer   10.43.131.244   172.19.0.7,172.19.0.8   8080:32125/TCP                          3h14m
istio-system      istiod                 ClusterIP      10.43.183.147   <none>                  15010/TCP,15012/TCP,443/TCP,15014/TCP   3h14m
k8gb              k8gb-coredns           ClusterIP      10.43.232.163   <none>                  53/UDP                                  3h14m
kube-system       kube-dns               ClusterIP      10.43.0.10      <none>                  53/UDP,53/TCP,9153/TCP                  3h15m
test-gslb-istio   frontend-podinfo       ClusterIP      10.43.111.176   <none>                  9898/TCP,9999/TCP                       3h14m
test-gslb-istio   unhealthy-app          ClusterIP      10.43.242.38    <none>                  80/TCP                                  3h14m
test-gslb         frontend-podinfo       ClusterIP      10.43.173.240   <none>                  9898/TCP,9999/TCP                       3h14m
test-gslb         unhealthy-app          ClusterIP      10.43.165.178   <none>                  80/TCP                                  3h14m

My understanding so far is that k3d is being used to run a few k3s clusters in Docker containers. I usually use kind for this. This is my first time using k3d. I understand with kind, if you're using Ingress, there is some setup involved to forward ports to your local machine. I'm wondering if that's where this might be breaking down. Perhaps this is required with the way we're using k3d in this tutorial and there's something wrong with that configuration. That's just my guess so far though.

More info about my setup:

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 24.04.1 LTS
Release:	24.04
Codename:	noble

$ kubectl version
Client Version: v1.32.1
Kustomize Version: v5.5.0
Server Version: v1.30.2+k3s1
WARNING: version difference between client (1.32) and server (1.30) exceeds the supported minor version skew of +/-1

$ helm version
version.BuildInfo{Version:"v3.17.0", GitCommit:"301108edc7ac2a8ba79e4ebf5701b0b6ce6a31e4", GitTreeState:"clean", GoVersion:"go1.23.4"}

$ docker version
Client: Docker Engine - Community
 Version:           27.5.1
 API version:       1.47
 Go version:        go1.22.11
 Git commit:        9f9e405
 Built:             Wed Jan 22 13:41:48 2025
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          27.5.1
  API version:      1.47 (minimum version 1.24)
  Go version:       go1.22.11
  Git commit:       4c9b3b0
  Built:            Wed Jan 22 13:41:48 2025
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.7.25
  GitCommit:        bcc810d6b9066471b0b6fa75f557a15a1cbf31bb
 runc:
  Version:          1.2.4
  GitCommit:        v1.2.4-0-g6c52b3f
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

$ k3d version
k3d version v5.8.1
k3s version v1.31.4-k3s1 (default)

$ golangci-lint version
golangci-lint has version 1.62.0 built with go1.23.2 from 22b58c9b on 2024-11-10T19:09:02Z
@k0da
Copy link
Collaborator

k0da commented Feb 11, 2025

@mattwelke by default local setup configures only UDP port to be forwarded through ingress into CoreDNS svc. I noticed you use +tcp flag, which forces dig to sent DNS queries over TCP.

@mattwelke
Copy link
Author

That did the trick. So the solution is likely a docs update: #1832

@mattwelke mattwelke linked a pull request Feb 12, 2025 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(docs): fix dig commands for each CoreDNS instance in local tutorial mattwelke/k8gb
2 participants
@k0da @mattwelke