Request for documentation/guidance on setting up consul agents

[ahawtho]

We've been setting up consul in k8s, using a similar setup to what you have here, but how do you handle individual consul agents? Using the pod abstraction, it would seems we'd have an agent per pod, which is how we've done it, but it inflates the deployment descriptor for each pod, couples the pod description to the consul server and imposes unnecessary overhead on the consul servers and on resources in the k8s cluster. It seems it would be more efficient to associate a single consul agent per node. I looked into DaemonSet to implement that, but there didn't seem to be a convenient way to associate the pods on the node with that node's agent. Have you run into this, or is 1:1 agent/pod the way to go?

The documentation in DaemonSet suggests the following without guidance:

Clients know the list of nodes ips somehow, and know the port by convention.

I did a PoC of this, attempting to provide the hostIP to a pod via the downward API, but hostIP isn't supported.

It seems in order to use DaemonSet, one would have to write an additional tool to watch the apiserver for pod creation events and then register each pod with its node's consul agent. It doesn't seem too hard, but I thought there might be a simpler way.

[dlaidlaw]

As of Kubernetes 1.7 the status.hostIP is available in the downward API.