LB ports do not work with external containerd. #11344

Magicloud · 2024-11-19T14:22:20Z

Magicloud
Nov 19, 2024

I need to run K3S with ZFS backend, hence I followed the doc here. Everything worked except that the ports exposed by service type load balancer were not connectable. The svclb pods were running.

I thought it might be related to OS and ZFS, so I started an Archlinux VM in VirtualBox. For Containerd, I changed nothing but

[plugins.'io.containerd.cri.v1.runtime'.cni]
  bin_dir = '/var/lib/rancher/k3s/data/current/bin/'
  conf_dir = '/var/lib/rancher/k3s/agent/etc/cni/net.d/'
[plugins.'io.containerd.grpc.v1.cri'.cni]
  bin_dir = '/var/lib/rancher/k3s/data/current/bin/'
  conf_dir = '/var/lib/rancher/k3s/agent/etc/cni/net.d/'

And installed K3S via curl -L https://get.k3s.io | CONTAINERD_HTTPS_PROXY=http://internal-proxy/ K3S_KUBECONFIG_MODE="644" INSTALL_K3S_EXEC="--container-runtime-endpoint unix:///run/containerd/containerd.sock" sh -. (I know the PROXY env would not work here, just keep things as I did.)

Same thing happened. The services are inaccessible. I do not have firewall or similar things running.

I posted the Iptables rules below. And tools info here:

kubectl version
Client Version: v1.30.6+k3s1
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.30.6+k3s1

containerd version
Server:
containerd:
Version: v2.0.0
runc:
Version: 1.2.2

I tried to directly running something with nerdctl, the port mapping worked. And the same Kubernetes manifests worked in another host running K3S with built-in containerd.

Magicloud · 2024-11-19T14:22:45Z

Magicloud
Nov 19, 2024
Author

# Generated by iptables-save v1.8.10 on Tue Nov 19 22:16:24 2024
*filter
:INPUT ACCEPT [5996:1610599]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [6358:1374286]
:FLANNEL-FWD - [0:0]
:KUBE-EXTERNAL-SERVICES - [0:0]
:KUBE-FIREWALL - [0:0]
:KUBE-FORWARD - [0:0]
:KUBE-KUBELET-CANARY - [0:0]
:KUBE-NODEPORTS - [0:0]
:KUBE-NWPLCY-DEFAULT - [0:0]
:KUBE-PROXY-CANARY - [0:0]
:KUBE-PROXY-FIREWALL - [0:0]
:KUBE-ROUTER-FORWARD - [0:0]
:KUBE-ROUTER-INPUT - [0:0]
:KUBE-ROUTER-OUTPUT - [0:0]
:KUBE-SERVICES - [0:0]
-A INPUT -m comment --comment "kube-router netpol - 4IA2OSFRMVNDXBVV" -j KUBE-ROUTER-INPUT
-A INPUT -j KUBE-FIREWALL
-A INPUT -m conntrack --ctstate NEW -m comment --comment "kubernetes load balancer firewall" -j KUBE-PROXY-FIREWALL
-A INPUT -m comment --comment "kubernetes health check service ports" -j KUBE-NODEPORTS
-A INPUT -m conntrack --ctstate NEW -m comment --comment "kubernetes externally-visible service portals" -j KUBE-EXTERNAL-SERVICES
-A INPUT -m comment --comment "KUBE-ROUTER rule to explicitly ACCEPT traffic that comply to network policies" -m mark --mark 0x20000/0x20000 -j ACCEPT
-A FORWARD -m comment --comment "kube-router netpol - TEMCG2JMHZYE7H7T" -j KUBE-ROUTER-FORWARD
-A FORWARD -m conntrack --ctstate NEW -m comment --comment "kubernetes load balancer firewall" -j KUBE-PROXY-FIREWALL
-A FORWARD -m comment --comment "kubernetes forwarding rules" -j KUBE-FORWARD
-A FORWARD -m conntrack --ctstate NEW -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
-A FORWARD -m conntrack --ctstate NEW -m comment --comment "kubernetes externally-visible service portals" -j KUBE-EXTERNAL-SERVICES
-A FORWARD -m comment --comment "KUBE-ROUTER rule to explicitly ACCEPT traffic that comply to network policies" -m mark --mark 0x20000/0x20000 -j ACCEPT
-A FORWARD -m comment --comment "flanneld forward" -j FLANNEL-FWD
-A OUTPUT -m comment --comment "kube-router netpol - VEAAIY32XVBHCSCY" -j KUBE-ROUTER-OUTPUT
-A OUTPUT -j KUBE-FIREWALL
-A OUTPUT -m conntrack --ctstate NEW -m comment --comment "kubernetes load balancer firewall" -j KUBE-PROXY-FIREWALL
-A OUTPUT -m conntrack --ctstate NEW -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
-A OUTPUT -m comment --comment "KUBE-ROUTER rule to explicitly ACCEPT traffic that comply to network policies" -m mark --mark 0x20000/0x20000 -j ACCEPT
-A FLANNEL-FWD -s 10.42.0.0/16 -m comment --comment "flanneld forward" -j ACCEPT
-A FLANNEL-FWD -d 10.42.0.0/16 -m comment --comment "flanneld forward" -j ACCEPT
-A KUBE-FIREWALL ! -s 127.0.0.0/8 -d 127.0.0.0/8 -m comment --comment "block incoming localnet connections" -m conntrack ! --ctstate RELATED,ESTABLISHED,DNAT -j DROP
-A KUBE-FORWARD -m conntrack --ctstate INVALID -j DROP
-A KUBE-FORWARD -m comment --comment "kubernetes forwarding rules" -m mark --mark 0x4000/0x4000 -j ACCEPT
-A KUBE-FORWARD -m comment --comment "kubernetes forwarding conntrack rule" -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A KUBE-NWPLCY-DEFAULT -p icmp -m comment --comment "allow icmp echo requests" -m icmp --icmp-type 8 -j ACCEPT
-A KUBE-NWPLCY-DEFAULT -p icmp -m comment --comment "allow icmp destination unreachable messages" -m icmp --icmp-type 3 -j ACCEPT
-A KUBE-NWPLCY-DEFAULT -p icmp -m comment --comment "allow icmp time exceeded messages" -m icmp --icmp-type 11 -j ACCEPT
-A KUBE-NWPLCY-DEFAULT -m comment --comment "rule to mark traffic matching a network policy" -j MARK --set-xmark 0x10000/0x10000
-A KUBE-ROUTER-INPUT -d 10.43.0.0/16 -m comment --comment "allow traffic to primary/secondary cluster IP range - EKROEGTNIJ3AP3LC" -j RETURN
-A KUBE-ROUTER-INPUT -p tcp -m comment --comment "allow LOCAL TCP traffic to node ports - LR7XO7NXDBGQJD2M" -m addrtype --dst-type LOCAL -m multiport --dports 30000:32767 -j RETURN
-A KUBE-ROUTER-INPUT -p udp -m comment --comment "allow LOCAL UDP traffic to node ports - 76UCBPIZNGJNWNUZ" -m addrtype --dst-type LOCAL -m multiport --dports 30000:32767 -j RETURN
COMMIT
# Completed on Tue Nov 19 22:16:24 2024
# Generated by iptables-save v1.8.10 on Tue Nov 19 22:16:24 2024
*mangle
:PREROUTING ACCEPT [75807:38003175]
:INPUT ACCEPT [72994:37605250]
:FORWARD ACCEPT [2813:397925]
:OUTPUT ACCEPT [75019:36487010]
:POSTROUTING ACCEPT [77832:36884935]
:KUBE-IPTABLES-HINT - [0:0]
:KUBE-KUBELET-CANARY - [0:0]
:KUBE-PROXY-CANARY - [0:0]
COMMIT
# Completed on Tue Nov 19 22:16:24 2024
# Generated by iptables-save v1.8.10 on Tue Nov 19 22:16:24 2024
*nat
:PREROUTING ACCEPT [20:2220]
:INPUT ACCEPT [24:2369]
:OUTPUT ACCEPT [2956:177959]
:POSTROUTING ACCEPT [3187:192526]
:CNI-DN-8de1f2de58b84b23c2356 - [0:0]
:CNI-DN-c5a932659fe7412720473 - [0:0]
:CNI-DN-db6928bf569b6578806c2 - [0:0]
:CNI-HOSTPORT-DNAT - [0:0]
:CNI-HOSTPORT-MASQ - [0:0]
:CNI-HOSTPORT-SETMARK - [0:0]
:FLANNEL-POSTRTG - [0:0]
:KUBE-EXT-CVG3OEGEH7H5P3HQ - [0:0]
:KUBE-EXT-DLB4IXM2MDEEENSQ - [0:0]
:KUBE-EXT-UNTZ66ODBJ7KDG2T - [0:0]
:KUBE-EXT-UQMCRMJZLI3FTLDP - [0:0]
:KUBE-KUBELET-CANARY - [0:0]
:KUBE-MARK-MASQ - [0:0]
:KUBE-NODEPORTS - [0:0]
:KUBE-POSTROUTING - [0:0]
:KUBE-PROXY-CANARY - [0:0]
:KUBE-SEP-BGJGUHJHLSFV5IK6 - [0:0]
:KUBE-SEP-BXF3PJKDKFLO2EPE - [0:0]
:KUBE-SEP-CAPCO5X2N23O3VV5 - [0:0]
:KUBE-SEP-DIM5C2R76ONRHF67 - [0:0]
:KUBE-SEP-EKQMUD6SWJUZGEU3 - [0:0]
:KUBE-SEP-JDOFMCL32JKLX3NY - [0:0]
:KUBE-SEP-OHE2HGBNBLYLOXDO - [0:0]
:KUBE-SEP-PHVAPZAZNNBZRE4A - [0:0]
:KUBE-SEP-YWYQW33PTTPWYJK7 - [0:0]
:KUBE-SERVICES - [0:0]
:KUBE-SVC-CVG3OEGEH7H5P3HQ - [0:0]
:KUBE-SVC-DLB4IXM2MDEEENSQ - [0:0]
:KUBE-SVC-ERIFXISQEP7F7OF4 - [0:0]
:KUBE-SVC-JD5MR3NA4I4DYORP - [0:0]
:KUBE-SVC-NPX46M4PTMTKRN6Y - [0:0]
:KUBE-SVC-TCOU7JCQXEZGVUNU - [0:0]
:KUBE-SVC-UNTZ66ODBJ7KDG2T - [0:0]
:KUBE-SVC-UQMCRMJZLI3FTLDP - [0:0]
:KUBE-SVC-Z4ANX4WAEWEBLCTM - [0:0]
-A PREROUTING -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
-A PREROUTING -m addrtype --dst-type LOCAL -j CNI-HOSTPORT-DNAT
-A PREROUTING -m addrtype --dst-type LOCAL -j CNI-HOSTPORT-DNAT
-A PREROUTING -m addrtype --dst-type LOCAL -j CNI-HOSTPORT-DNAT
-A OUTPUT -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
-A OUTPUT -m addrtype --dst-type LOCAL -j CNI-HOSTPORT-DNAT
-A OUTPUT -m addrtype --dst-type LOCAL -j CNI-HOSTPORT-DNAT
-A POSTROUTING -m comment --comment "CNI portfwd requiring masquerade" -j CNI-HOSTPORT-MASQ
-A POSTROUTING -m comment --comment "CNI portfwd requiring masquerade" -j CNI-HOSTPORT-MASQ
-A POSTROUTING -m comment --comment "kubernetes postrouting rules" -j KUBE-POSTROUTING
-A POSTROUTING -m comment --comment "flanneld masq" -j FLANNEL-POSTRTG
-A CNI-DN-8de1f2de58b84b23c2356 -s 10.42.0.0/24 -p tcp -m tcp --dport 8081 -j CNI-HOSTPORT-SETMARK
-A CNI-DN-8de1f2de58b84b23c2356 -s 127.0.0.1/32 -p tcp -m tcp --dport 8081 -j CNI-HOSTPORT-SETMARK
-A CNI-DN-8de1f2de58b84b23c2356 -p tcp -m tcp --dport 8081 -j DNAT --to-destination 10.42.0.87:8081
-A CNI-DN-c5a932659fe7412720473 -s 10.42.0.0/24 -p tcp -m tcp --dport 2081 -j CNI-HOSTPORT-SETMARK
-A CNI-DN-c5a932659fe7412720473 -s 127.0.0.1/32 -p tcp -m tcp --dport 2081 -j CNI-HOSTPORT-SETMARK
-A CNI-DN-c5a932659fe7412720473 -p tcp -m tcp --dport 2081 -j DNAT --to-destination 10.42.0.85:2081
-A CNI-DN-db6928bf569b6578806c2 -s 10.42.0.0/24 -p tcp -m tcp --dport 80 -j CNI-HOSTPORT-SETMARK
-A CNI-DN-db6928bf569b6578806c2 -s 127.0.0.1/32 -p tcp -m tcp --dport 80 -j CNI-HOSTPORT-SETMARK
-A CNI-DN-db6928bf569b6578806c2 -p tcp -m tcp --dport 80 -j DNAT --to-destination 10.42.0.86:80
-A CNI-DN-db6928bf569b6578806c2 -s 10.42.0.0/24 -p tcp -m tcp --dport 443 -j CNI-HOSTPORT-SETMARK
-A CNI-DN-db6928bf569b6578806c2 -s 127.0.0.1/32 -p tcp -m tcp --dport 443 -j CNI-HOSTPORT-SETMARK
-A CNI-DN-db6928bf569b6578806c2 -p tcp -m tcp --dport 443 -j DNAT --to-destination 10.42.0.86:443
-A CNI-HOSTPORT-DNAT -p tcp -m comment --comment "dnat name: \"cbr0\" id: \"c74eadefdacfc338d43526cca6958f636ac657cc528f14048eac1c030a0973bc\"" -m multiport --dports 8081 -j CNI-DN-8de1f2de58b84b23c2356
-A CNI-HOSTPORT-DNAT -p tcp -m comment --comment "dnat name: \"cbr0\" id: \"425e1c84f3b41c5186a97f14c7a5bd927f824c1b512e11a31aba413eee0b954a\"" -m multiport --dports 2081 -j CNI-DN-c5a932659fe7412720473
-A CNI-HOSTPORT-DNAT -p tcp -m comment --comment "dnat name: \"cbr0\" id: \"dd37016c1f7d542763c332cbcb9de50be658a3f1360b55b0940059af30f3c436\"" -m multiport --dports 80,443 -j CNI-DN-db6928bf569b6578806c2
-A CNI-HOSTPORT-MASQ -m mark --mark 0x2000/0x2000 -j MASQUERADE
-A CNI-HOSTPORT-MASQ -m mark --mark 0x2000/0x2000 -j MASQUERADE
-A CNI-HOSTPORT-SETMARK -m comment --comment "CNI portfwd masquerade mark" -j MARK --set-xmark 0x2000/0x2000
-A FLANNEL-POSTRTG -m mark --mark 0x4000/0x4000 -m comment --comment "flanneld masq" -j RETURN
-A FLANNEL-POSTRTG -s 10.42.0.0/24 -d 10.42.0.0/16 -m comment --comment "flanneld masq" -j RETURN
-A FLANNEL-POSTRTG -s 10.42.0.0/16 -d 10.42.0.0/24 -m comment --comment "flanneld masq" -j RETURN
-A FLANNEL-POSTRTG ! -s 10.42.0.0/16 -d 10.42.0.0/24 -m comment --comment "flanneld masq" -j RETURN
-A FLANNEL-POSTRTG -s 10.42.0.0/16 ! -d 224.0.0.0/4 -m comment --comment "flanneld masq" -j MASQUERADE --random-fully
-A FLANNEL-POSTRTG ! -s 10.42.0.0/16 -d 10.42.0.0/16 -m comment --comment "flanneld masq" -j MASQUERADE --random-fully
-A KUBE-EXT-CVG3OEGEH7H5P3HQ -m comment --comment "masquerade traffic for kube-system/traefik:websecure external destinations" -j KUBE-MARK-MASQ
-A KUBE-EXT-CVG3OEGEH7H5P3HQ -j KUBE-SVC-CVG3OEGEH7H5P3HQ
-A KUBE-EXT-DLB4IXM2MDEEENSQ -m comment --comment "masquerade traffic for proxy/socks_proxy external destinations" -j KUBE-MARK-MASQ
-A KUBE-EXT-DLB4IXM2MDEEENSQ -j KUBE-SVC-DLB4IXM2MDEEENSQ
-A KUBE-EXT-UNTZ66ODBJ7KDG2T -m comment --comment "masquerade traffic for proxy/socks2http external destinations" -j KUBE-MARK-MASQ
-A KUBE-EXT-UNTZ66ODBJ7KDG2T -j KUBE-SVC-UNTZ66ODBJ7KDG2T
-A KUBE-EXT-UQMCRMJZLI3FTLDP -m comment --comment "masquerade traffic for kube-system/traefik:web external destinations" -j KUBE-MARK-MASQ
-A KUBE-EXT-UQMCRMJZLI3FTLDP -j KUBE-SVC-UQMCRMJZLI3FTLDP
-A KUBE-MARK-MASQ -j MARK --set-xmark 0x4000/0x4000
-A KUBE-NODEPORTS -p tcp -m comment --comment "kube-system/traefik:websecure" -m tcp --dport 30829 -j KUBE-EXT-CVG3OEGEH7H5P3HQ
-A KUBE-NODEPORTS -p tcp -m comment --comment "proxy/socks2http" -m tcp --dport 30853 -j KUBE-EXT-UNTZ66ODBJ7KDG2T
-A KUBE-NODEPORTS -p tcp -m comment --comment "kube-system/traefik:web" -m tcp --dport 31772 -j KUBE-EXT-UQMCRMJZLI3FTLDP
-A KUBE-NODEPORTS -p tcp -m comment --comment "proxy/socks_proxy" -m tcp --dport 30813 -j KUBE-EXT-DLB4IXM2MDEEENSQ
-A KUBE-POSTROUTING -m mark ! --mark 0x4000/0x4000 -j RETURN
-A KUBE-POSTROUTING -j MARK --set-xmark 0x4000/0x0
-A KUBE-POSTROUTING -m comment --comment "kubernetes service traffic requiring SNAT" -j MASQUERADE --random-fully
-A KUBE-SEP-BGJGUHJHLSFV5IK6 -s 10.42.0.91/32 -m comment --comment "kube-system/traefik:websecure" -j KUBE-MARK-MASQ
-A KUBE-SEP-BGJGUHJHLSFV5IK6 -p tcp -m comment --comment "kube-system/traefik:websecure" -m tcp -j DNAT --to-destination 10.42.0.91:8443
-A KUBE-SEP-BXF3PJKDKFLO2EPE -s 10.42.0.91/32 -m comment --comment "kube-system/traefik:web" -j KUBE-MARK-MASQ
-A KUBE-SEP-BXF3PJKDKFLO2EPE -p tcp -m comment --comment "kube-system/traefik:web" -m tcp -j DNAT --to-destination 10.42.0.91:8000
-A KUBE-SEP-CAPCO5X2N23O3VV5 -s 10.42.0.93/32 -m comment --comment "proxy/socks_proxy" -j KUBE-MARK-MASQ
-A KUBE-SEP-CAPCO5X2N23O3VV5 -p tcp -m comment --comment "proxy/socks_proxy" -m tcp -j DNAT --to-destination 10.42.0.93:1080
-A KUBE-SEP-DIM5C2R76ONRHF67 -s 10.42.0.88/32 -m comment --comment "kube-system/kube-dns:dns-tcp" -j KUBE-MARK-MASQ
-A KUBE-SEP-DIM5C2R76ONRHF67 -p tcp -m comment --comment "kube-system/kube-dns:dns-tcp" -m tcp -j DNAT --to-destination 10.42.0.88:53
-A KUBE-SEP-EKQMUD6SWJUZGEU3 -s 10.42.0.88/32 -m comment --comment "kube-system/kube-dns:metrics" -j KUBE-MARK-MASQ
-A KUBE-SEP-EKQMUD6SWJUZGEU3 -p tcp -m comment --comment "kube-system/kube-dns:metrics" -m tcp -j DNAT --to-destination 10.42.0.88:9153
-A KUBE-SEP-JDOFMCL32JKLX3NY -s 10.42.0.92/32 -m comment --comment "proxy/socks2http" -j KUBE-MARK-MASQ
-A KUBE-SEP-JDOFMCL32JKLX3NY -p tcp -m comment --comment "proxy/socks2http" -m tcp -j DNAT --to-destination 10.42.0.92:8123
-A KUBE-SEP-OHE2HGBNBLYLOXDO -s 10.42.0.88/32 -m comment --comment "kube-system/kube-dns:dns" -j KUBE-MARK-MASQ
-A KUBE-SEP-OHE2HGBNBLYLOXDO -p udp -m comment --comment "kube-system/kube-dns:dns" -m udp -j DNAT --to-destination 10.42.0.88:53
-A KUBE-SEP-PHVAPZAZNNBZRE4A -s 192.168.0.121/32 -m comment --comment "default/kubernetes:https" -j KUBE-MARK-MASQ
-A KUBE-SEP-PHVAPZAZNNBZRE4A -p tcp -m comment --comment "default/kubernetes:https" -m tcp -j DNAT --to-destination 192.168.0.121:6443
-A KUBE-SEP-YWYQW33PTTPWYJK7 -s 10.42.0.89/32 -m comment --comment "kube-system/metrics-server:https" -j KUBE-MARK-MASQ
-A KUBE-SEP-YWYQW33PTTPWYJK7 -p tcp -m comment --comment "kube-system/metrics-server:https" -m tcp -j DNAT --to-destination 10.42.0.89:10250
-A KUBE-SERVICES -d 10.43.236.6/32 -p tcp -m comment --comment "kube-system/traefik:websecure cluster IP" -m tcp --dport 443 -j KUBE-SVC-CVG3OEGEH7H5P3HQ
-A KUBE-SERVICES -d 192.168.0.121/32 -p tcp -m comment --comment "kube-system/traefik:websecure loadbalancer IP" -m tcp --dport 443 -j KUBE-EXT-CVG3OEGEH7H5P3HQ
-A KUBE-SERVICES -d 10.43.48.118/32 -p tcp -m comment --comment "proxy/socks2http cluster IP" -m tcp --dport 8081 -j KUBE-SVC-UNTZ66ODBJ7KDG2T
-A KUBE-SERVICES -d 192.168.0.121/32 -p tcp -m comment --comment "proxy/socks2http loadbalancer IP" -m tcp --dport 8081 -j KUBE-EXT-UNTZ66ODBJ7KDG2T
-A KUBE-SERVICES -d 10.43.0.10/32 -p udp -m comment --comment "kube-system/kube-dns:dns cluster IP" -m udp --dport 53 -j KUBE-SVC-TCOU7JCQXEZGVUNU
-A KUBE-SERVICES -d 10.43.0.10/32 -p tcp -m comment --comment "kube-system/kube-dns:metrics cluster IP" -m tcp --dport 9153 -j KUBE-SVC-JD5MR3NA4I4DYORP
-A KUBE-SERVICES -d 10.43.202.143/32 -p tcp -m comment --comment "kube-system/metrics-server:https cluster IP" -m tcp --dport 443 -j KUBE-SVC-Z4ANX4WAEWEBLCTM
-A KUBE-SERVICES -d 10.43.236.6/32 -p tcp -m comment --comment "kube-system/traefik:web cluster IP" -m tcp --dport 80 -j KUBE-SVC-UQMCRMJZLI3FTLDP
-A KUBE-SERVICES -d 192.168.0.121/32 -p tcp -m comment --comment "kube-system/traefik:web loadbalancer IP" -m tcp --dport 80 -j KUBE-EXT-UQMCRMJZLI3FTLDP
-A KUBE-SERVICES -d 10.43.18.142/32 -p tcp -m comment --comment "proxy/socks_proxy cluster IP" -m tcp --dport 2081 -j KUBE-SVC-DLB4IXM2MDEEENSQ
-A KUBE-SERVICES -d 192.168.0.121/32 -p tcp -m comment --comment "proxy/socks_proxy loadbalancer IP" -m tcp --dport 2081 -j KUBE-EXT-DLB4IXM2MDEEENSQ
-A KUBE-SERVICES -d 10.43.0.1/32 -p tcp -m comment --comment "default/kubernetes:https cluster IP" -m tcp --dport 443 -j KUBE-SVC-NPX46M4PTMTKRN6Y
-A KUBE-SERVICES -d 10.43.0.10/32 -p tcp -m comment --comment "kube-system/kube-dns:dns-tcp cluster IP" -m tcp --dport 53 -j KUBE-SVC-ERIFXISQEP7F7OF4
-A KUBE-SERVICES -m comment --comment "kubernetes service nodeports; NOTE: this must be the last rule in this chain" -m addrtype --dst-type LOCAL -j KUBE-NODEPORTS
-A KUBE-SVC-CVG3OEGEH7H5P3HQ ! -s 10.42.0.0/16 -d 10.43.236.6/32 -p tcp -m comment --comment "kube-system/traefik:websecure cluster IP" -m tcp --dport 443 -j KUBE-MARK-MASQ
-A KUBE-SVC-CVG3OEGEH7H5P3HQ -m comment --comment "kube-system/traefik:websecure -> 10.42.0.91:8443" -j KUBE-SEP-BGJGUHJHLSFV5IK6
-A KUBE-SVC-DLB4IXM2MDEEENSQ ! -s 10.42.0.0/16 -d 10.43.18.142/32 -p tcp -m comment --comment "proxy/socks_proxy cluster IP" -m tcp --dport 2081 -j KUBE-MARK-MASQ
-A KUBE-SVC-DLB4IXM2MDEEENSQ -m comment --comment "proxy/socks_proxy -> 10.42.0.93:1080" -j KUBE-SEP-CAPCO5X2N23O3VV5
-A KUBE-SVC-ERIFXISQEP7F7OF4 ! -s 10.42.0.0/16 -d 10.43.0.10/32 -p tcp -m comment --comment "kube-system/kube-dns:dns-tcp cluster IP" -m tcp --dport 53 -j KUBE-MARK-MASQ
-A KUBE-SVC-ERIFXISQEP7F7OF4 -m comment --comment "kube-system/kube-dns:dns-tcp -> 10.42.0.88:53" -j KUBE-SEP-DIM5C2R76ONRHF67
-A KUBE-SVC-JD5MR3NA4I4DYORP ! -s 10.42.0.0/16 -d 10.43.0.10/32 -p tcp -m comment --comment "kube-system/kube-dns:metrics cluster IP" -m tcp --dport 9153 -j KUBE-MARK-MASQ
-A KUBE-SVC-JD5MR3NA4I4DYORP -m comment --comment "kube-system/kube-dns:metrics -> 10.42.0.88:9153" -j KUBE-SEP-EKQMUD6SWJUZGEU3
-A KUBE-SVC-NPX46M4PTMTKRN6Y ! -s 10.42.0.0/16 -d 10.43.0.1/32 -p tcp -m comment --comment "default/kubernetes:https cluster IP" -m tcp --dport 443 -j KUBE-MARK-MASQ
-A KUBE-SVC-NPX46M4PTMTKRN6Y -m comment --comment "default/kubernetes:https -> 192.168.0.121:6443" -j KUBE-SEP-PHVAPZAZNNBZRE4A
-A KUBE-SVC-TCOU7JCQXEZGVUNU ! -s 10.42.0.0/16 -d 10.43.0.10/32 -p udp -m comment --comment "kube-system/kube-dns:dns cluster IP" -m udp --dport 53 -j KUBE-MARK-MASQ
-A KUBE-SVC-TCOU7JCQXEZGVUNU -m comment --comment "kube-system/kube-dns:dns -> 10.42.0.88:53" -j KUBE-SEP-OHE2HGBNBLYLOXDO
-A KUBE-SVC-UNTZ66ODBJ7KDG2T ! -s 10.42.0.0/16 -d 10.43.48.118/32 -p tcp -m comment --comment "proxy/socks2http cluster IP" -m tcp --dport 8081 -j KUBE-MARK-MASQ
-A KUBE-SVC-UNTZ66ODBJ7KDG2T -m comment --comment "proxy/socks2http -> 10.42.0.92:8123" -j KUBE-SEP-JDOFMCL32JKLX3NY
-A KUBE-SVC-UQMCRMJZLI3FTLDP ! -s 10.42.0.0/16 -d 10.43.236.6/32 -p tcp -m comment --comment "kube-system/traefik:web cluster IP" -m tcp --dport 80 -j KUBE-MARK-MASQ
-A KUBE-SVC-UQMCRMJZLI3FTLDP -m comment --comment "kube-system/traefik:web -> 10.42.0.91:8000" -j KUBE-SEP-BXF3PJKDKFLO2EPE
-A KUBE-SVC-Z4ANX4WAEWEBLCTM ! -s 10.42.0.0/16 -d 10.43.202.143/32 -p tcp -m comment --comment "kube-system/metrics-server:https cluster IP" -m tcp --dport 443 -j KUBE-MARK-MASQ
-A KUBE-SVC-Z4ANX4WAEWEBLCTM -m comment --comment "kube-system/metrics-server:https -> 10.42.0.89:10250" -j KUBE-SEP-YWYQW33PTTPWYJK7
COMMIT
# Completed on Tue Nov 19 22:16:24 2024

4 replies

Magicloud Nov 19, 2024
Author

I have two ports exposed.

8081 for socks2http
2081 for socks_proxy

brandond Nov 19, 2024
Collaborator

Have you tried with containerd 1.7? 2.0 just came out fairly recently.

Magicloud Nov 19, 2024
Author

I used 1.7 in NixOS with ZFS, did not work. Trying to see if I can get 1.7 in Archlinux.

brandond Nov 19, 2024
Collaborator

This is all just kubelet stuff, I don't see how the container runtime would make a difference.

Magicloud · 2024-11-19T17:36:34Z

Magicloud
Nov 19, 2024
Author

There must be some mistake in my process. I reworked all things and now it works.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

LB ports do not work with external containerd. #11344

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 2 comments 4 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

LB ports do not work with external containerd. #11344

Magicloud Nov 19, 2024

Replies: 2 comments · 4 replies

Magicloud Nov 19, 2024 Author

Magicloud Nov 19, 2024 Author

brandond Nov 19, 2024 Collaborator

Magicloud Nov 19, 2024 Author

brandond Nov 19, 2024 Collaborator

Magicloud Nov 19, 2024 Author

Magicloud
Nov 19, 2024

Replies: 2 comments 4 replies

Magicloud
Nov 19, 2024
Author

Magicloud Nov 19, 2024
Author

brandond Nov 19, 2024
Collaborator

Magicloud Nov 19, 2024
Author

brandond Nov 19, 2024
Collaborator

Magicloud
Nov 19, 2024
Author