[draft] expose coredns as loadbalancer service in local setup

[abaguas]

Before this PR, the local setup was exposing coreDNS using a nodeport service. To expose it outside of k3d it required mapping the exposed DNS port 5053/5054 (for cluster1 and cluster2, respectively) to a single node and making sure coredns is running there. This is a bit hacky and not the usual way to expose a service outside of a cluster.

k get svc -n k8gb
NAME               TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)         AGE
...
k8gb-coredns-tcp   NodePort    10.43.37.190   <none>        53:30053/TCP    117s

This PR changes the setup to using the more common loadbalancer service, obtaining the same result with a bit less configuration.

[k3d-test-gslb1|k8gb] ➜  k get svc k8gb-coredns
NAME           TYPE           CLUSTER-IP     EXTERNAL-IP             PORT(S)                     AGE
k8gb-coredns   LoadBalancer   10.43.59.251   172.18.0.6,172.18.0.7   53:32301/UDP,53:32301/TCP   16m

Additionally, DNS queries to the clusters now work using UDP and TCP, before only TCP was possible:

➜  ~ dig -p 5053 +tcp @localhost localtargets-failover.cloud.example.com +short
172.18.0.6
172.18.0.7
➜  ~ dig -p 5053 @localhost localtargets-failover.cloud.example.com +short
172.18.0.6
172.18.0.7
➜  ~ dig -p 5054 @localhost localtargets-failover.cloud.example.com +short
172.18.0.11
172.18.0.12
➜  ~ dig -p 5054 +tcp @localhost localtargets-failover.cloud.example.com +short
172.18.0.11
172.18.0.12

---

Note: k3d behaves a bit different than the usual kubernetes cluster since it uses a daemon set behind the scenes. That is why the load balancer has multiple external IPs. Nonetheless, the functionality for the user is the exact same.

[k3d-test-gslb1|k8gb] ➜  ~/code/k8gb git:(coredns/loadbalancer) kgp -n kube-system -o wide | grep coredns
svclb-k8gb-coredns-15f61200-ljxxx           2/2     Running   0          18m   10.42.1.3   k3d-test-gslb1-server-0   <none>           <none>
svclb-k8gb-coredns-15f61200-vcz6l           2/2     Running   0          18m   10.42.0.2   k3d-test-gslb1-agent-0    <none>           <none>

[k0da]

@abaguas any blockers with this?