[tempo] bugfix: Service name is different from the Service name specified in …

[tozastation]

#3042

Problem Detail:

• StatefulSets resource defines serviceName as ${SERVICE_NAME}-headless
• However service name is defined in ${SERVICE_NAME}.
• So, name resolution with FQDN in Headless Service is not possible.

Resolution:

• Assign a headless suffix to the Service resource name
• or
• Remove Headless prefix from Service Name in StatefulSet

Reference:

• StatefulSet (spec.serviceName)
  • https://github.com/grafana/helm-charts/blob/main/charts/tempo/templates/statefulset.yaml#L20
• Service (metadata.name)
  • https://github.com/grafana/helm-charts/blob/main/charts/tempo/templates/service.yaml#L4

[CLAassistant]

All committers have signed the CLA.

[tozastation]

Comment: This is because Helm Values can be set even if the Service resource is not Headless.

[dastrobu]

It would be better if serviceName refers to a headless service. Currently, {{ template "tempo.fullname" . }} is not referring to a headless service. Therefore, there might be some additional changes required. However, I completely agree that this should be cleaned up.

[tozastation]

@dastrobu
thanks! I have changed the suffix to be consistent with headless.
I just want to confirm one point: do you think the cluster ip value needs to be changed to default to None?
ref: https://github.com/grafana/helm-charts/blob/main/charts/tempo/templates/service.yaml#L16

[dastrobu]

@tozastation note that I am not a maintainer of Grafana Tempo. I just ran into the same issue looking for a fix.

Personally, I'd keep the service name (to avoid a breaking change on updating the chart for users).

There are two potential alternatives to fix this from my point of view:

1. add a headless service called {{ template "tempo.fullname" . }}-headless but keep the original service in addition. The headless service would then be used for member list joining, the other one for pushing traces from applications.
2. change the current service to be headless (clusterIP: None) and refer to it from the stateful set. I am not sure what implications it will have on clients pushing traces if the services is headless - as they will see multiple DNS records for the service.

I think someone from the maintainers should comment on the direction to go.