parse error in grafana-k8s unit's charm container

[agileshaw]

Grafana-k8s reports parse error upon unit start in its charm container:

2024-02-14T15:52:25.455Z [pebble] Check "readiness" failure 1 (threshold 3): received non-20x status code 418
2024-02-14T15:52:27.208Z [container-agent] 2024-02-14 15:52:27 WARNING upgrade-charm time="2024-02-14T15:52:27Z" level=fatal msg="1:1: parse error: no expression found in input"
2024-02-14T15:52:27.498Z [container-agent] 2024-02-14 15:52:27 WARNING upgrade-charm time="2024-02-14T15:52:27Z" level=fatal msg="1:115: parse error: unexpected character: '$'"
2024-02-14T15:52:27.717Z [container-agent] 2024-02-14 15:52:27 WARNING upgrade-charm time="2024-02-14T15:52:27Z" level=fatal msg="1:13: parse error: unexpected character: '$'"
2024-02-14T15:52:27.770Z [container-agent] 2024-02-14 15:52:27 WARNING upgrade-charm time="2024-02-14T15:52:27Z" level=fatal msg="1:32: parse error: unexpected character: '$'"

The bundle I observed the issue in is https://paste.ubuntu.com/p/kp9RdjxNDc/, deployed as part of a Sunbeam 2023/edge when enabling observability plugin.

[sed-i]

Could you please provide the names of the charms that juju obfuscated in the bundle?

- - grafana:grafana-dashboard
  - remote-500ea25c3e674973886d218b73106005:grafana-dashboards-provider
- - grafana:grafana-dashboard
  - remote-7dd10d7de5ff44e6830ca010acc213eb:grafana-dashboards-provider

[agileshaw]

It's grafana-k8s, revision 51

[sed-i]

I meant what charms forward dashboards to grafana? What do you have on the other side of the grafana-dashboard CMR?
If it's grafana-agent, then which charms' dashboards are being collected?

[agileshaw]

I see. It's tempest-k8s built on top of this patch

[mmkay]

Grafana version that was used here is from December 2022 - our current revision is rev111. We'll try to reproduce the issue with the provided dashboard using the latest revision.