Configuration of trusted_proxies for deployment behind a reverse-proxy

[aharabedian]

Is there a way to pass customize the configuration.yaml that is deployed in the chart to include values for trusted_proxies? This would be very useful for setups that are using a reverse-proxy.

If possible, adding an additional configuration file that is auto filled with appropriate values if a reverse proxy is in use would be an excellent addition. This file could be included from the base configuration.yaml file.

[pajikos]

Hi,
In theory, I could add a default configuration to the Helm chart, but it would only apply once, before the initial start of Home Assistant. Editing the configuration of an already running Home Assistant via Helm chart upgrade is not a good idea. It could potentially cause problems with Home Assistant.

The second issue is related to the values within trusted_proxies. It should be the IP address of the reverse proxy, but obtaining this value during the Helm chart installation is not feasible. Installing a reverse proxy is beyond the scope of this Helm chart.

[jessebot]

Sorry to bother on this thread, but was wondering what we should do if we're using nginx? Example values:

values.yaml

ingress:
  enabled: true
  className: "nginx"
  annotations:
    kubernetes.io/tls-acme: "true"
    cert-manager.io/cluster-issuer: letsencrypt-staging
  hosts:
    - host: home-assistant.beep.boop
      paths:
        - path: /
          pathType: ImplementationSpecific
  tls:
   - secretName: home-assistant-tls
     hosts:
       - home-assistant.beep.boop

I am trying right now to use the reverse proxy feature now since otherwise I get an error using the above values.yaml:

WARNING (MainThread) [homeassistant.components.http.forwarded] A request from a reverse proxy was received from 10.---.---.---, but your HTTP integration is not set-up for reverse proxies

My solution was going to be to use a ConfigMap to add a default configuration.yaml file based on this thread, but I get this error when I try:

Error: INSTALLATION FAILED: YAML parse error on home-assistant/templates/statefulset.yaml: error converting YAML to JSON: yaml: line 49: did not find expected key

helm command:

helm install home-assistant pajikos/home-assistant --namespace home-assistant --values values.yaml

values.yaml

additionalVolumes:
  - name: home-assistant-config
    configMap:
      name: home-assistant-config

additionalVolumeMounts:
  - name: home-assistant-config
    mountPath: "/config"

ingress:
  enabled: true
  className: "nginx"
  annotations:
    kubernetes.io/tls-acme: "true"
    cert-manager.io/cluster-issuer: letsencrypt-staging
  hosts:
    - host: home-assistant.beep.boop
      paths:
        - path: /
          pathType: ImplementationSpecific
  tls:
   - secretName: home-assistant-tls
     hosts:
       - home-assistant.beep.boop

configmap.yaml applied with kubectl

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: home-assistant-config
data:
  configuration.yaml: |
    http:
      use_x_forwarded_for: true
      trusted_proxies:
        - 10.0.0.0/8

I noticed that you have this volume mount already declared:

home-assistant-helm-chart/charts/home-assistant/templates/statefulset.yaml

Line 65 in 7ce22b7

- mountPath: /config

Could you perhaps make that optional? Then I think I could override the default with my own configmap.Also open to other suggestions!

Update

I tried changing the configmap name to the release name/fullnameoverride but still got the same error. Second attempt:

values.yaml

fullnameOverride: home-assistant

additionalVolumes:
  - name: home-assistant
    configMap:
      name: home-assistant

ingress:
  enabled: true
  className: "nginx"
  annotations:
    kubernetes.io/tls-acme: "true"
    cert-manager.io/cluster-issuer: letsencrypt-staging
  hosts:
    - host: home-assistant.beep.boop
      paths:
        - path: /
          pathType: ImplementationSpecific
  tls:
   - secretName: home-assistant-tls
     hosts:
       - home-assistant.beep.boop

ConfigMap - also trying with specific ip in case that affects something

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: home-assistant
data:
  configuration.yaml: |
    http:
      use_x_forwarded_for: true
      trusted_proxies:
        - 10.--.--.--

[pajikos]

Hi, I've identified two issues:

Firstly, try mounting only one file:

additionalVolumeMounts:
 - name: home-assistant-config
    mountPath: /config/configuration.yaml
    subPath: configuration.yaml

The existing /config mount is in place to share the same settings with the VSCode extension. However, I believe it's not a best practice to mount configuration.yaml directly into a running Home Assistant instance. When mounted from a ConfigMap, the configuration.yaml file becomes read-only, making it impossible to edit, even in VSCode.

Secondly, the trusted_proxies value should include the IP addresses of your Nginx controller. In a Kubernetes environment, this can be challenging to determine; however, when using k3s, I typically set the entire pod CIDR range to 10.42.0.0/16 (as per the k3s documentation on networking). For added security, I also implement a network policy, although this is optional.

[ytjohn]

Thanks for the helm chart. I ran into this as well.

My recommendation is for someone setting up the first time is to make use of port forwarding for the first time setup.

If you just need port forwarding on localhost:

kubectl -n homeassistant port-forward pod/helm-homeassistant-home-assistant-0 8123 8123

But in most cases, your k3s/k8s cluster is remote, so you need to listen all addresses:

kubectl -n homeassistant port-forward pod/helm-homeassistant-home-assistant-0 --address 0.0.0.0 8123 8123

`

[jlpedrosa]

@pajikos I also run into this. I am not sure if it would be a good idea to manage the /config/configuration.yaml mount and config map directly into the chart? that would allow you to at least have a "deploy" and works. On the downside, making backups is harder as just snapshotting the pvc is not enough. I'm a bit torn into this one.

[NikoKS]

@pajikos Any updates for solution on this topic?

[Starkie]

Hey guys!

I was able to make it work by editing the configuration.yaml in the pod itself. Since it is stored in the persistent volume, the changes will persist across restarts :)

We can get into the pod with the following commands:

kubectl exec -it -n home-assistant pod/home-assistant-0 -- bash

Now let's edit the config file:

vi configuration.yaml

Append this config at the end of the file, adapt the IP to your specific config. I'm using K3s with traefik as the reverse proxy:

http:
  use_x_forwarded_for: true
  trusted_proxies:
    - 10.42.0.0/16

Restart the statefulset to apply the changes:

 kubectl rollout restart -n home-assistant statefulset.apps/home-assistant                                                                                                 

Now you should be able to access the app using your ingress.

[jlpedrosa]

@Starkie Yeah, that is of course an option, but the idea of making the chart provide a working configuration I think is the best solution.

[jlpedrosa]

@pajikos IMO the golden solution for this have an init container that updates the config when a certain .Values.X is enabled probably with yq -i '.http.use_x_forwarded_for = true' configuration.yaml

I have a couple of complicated months ahead, I can't commit to fix this, but I can help review if you jump into it.

[pajikos]

@jlpedrosa ok, I will suggest such a solution, but I think, http.trusted_proxies is required as well and that is the problem, each cluster has its own IP ranges. Are you using http.use_x_forwarded_for only?

[RLutsch]

maybe something like this:

• in values file we have

configmap:
    configuration.yaml: | -
       stuff here 

as part of the chart
if config = true create the default configuration.yaml and mount it.

This way if you want to use vanilla version you leave out and if you want custom config you can override.

happy to do the pr

[pajikos]

maybe something like this:

• in values file we have

configmap:
    configuration.yaml: | -
       stuff here 

as part of the chart if config = true create the default configuration.yaml and mount it.

This way if you want to use vanilla version you leave out and if you want custom config you can override.

happy to do the pr

Be careful here, mounting the configuration.yaml is not a problem, but then it is not possible to edit this file. It would be necessary to mount this file into an init container and then copy it from there into the target configuration.yaml. I have already implemented a similar pattern on another project, I will try to outline it.

[dominch]

I've got same issue today and eventually I managed to solve that by changing service type to LoadBalancer:

service:
  # Service type (ClusterIP, NodePort, LoadBalancer, or ExternalName)
  type: LoadBalancer

With this small change I've got no more connection errors to this service and probably no need for config change in files.

edit: I created several headers middlewares earlier (in traefik), those work perfectly on GET, but still not for POST request, port forward has no such issue, still several things to modify to get it fully working on external domain via whitelist policy :/

[jlpedrosa]

Loadbancer may work as long as you don't use an ingress (depends on the loadbalancer), but with in an ingress it will always be L7 Proxy, and it will have the issue.

@RLutsch I agree with @pajikos, We need to be very careful so we don't override other settings managed by the user. That is why I was sugesting to do it item by item in yq.

yq -i '.http.use_x_forwarded_for = true' configuration.yaml
yq -i '.http.trusted_proxies = [ "10.42.0.0/16" ] configuration.yaml

this ensures that only those two keys are modified. cc: @pajikos I was using both, but I was puting one for simplicty

[jlpedrosa]

#54

[dominch]

Yes, I got to the point where most of request were working but still some post and background ajax queries returned error. It was configured with traefik, but still I need to modify several things to make it working as well as on port forwarding (no problem here, but no ssl).

[jjacobsson]

Hey, maybe this is incredibly dumb so feel free to ignore.

But would it not make sense to (if at all possible) add an include in configuration.yaml to some file... and have the helm-chart write said file, either making it just [] because it is empty or write whatever is found under a particular key in values.yaml like @RLutsch suggested above?

That way there would be a way to configure anything in home assistant by putting the relevant yaml in your own values for the chart... well... almost anything. You would still have problems if you wanted to remove something I guess.

I don't know, I just thought it would be "neat" to use an include file. Mostly because you would keep the changes to configuration.yaml as minimal as possible.

[pajikos]

Hi, I have developed a general approach for configuring Home Assistant, including the use of a reverse proxy. See #55 for details. @jlpedrosa or others, it's quite a significant change, so it would be fine to test it before merging. Can you help me with this? I'm particularly concerned about using securityContext at any level. Many thanks.

[jjacobsson]

I can if you can wait a bit, I am out of town until the weekend. tis 14 maj 2024 kl. 10:01 skrev Pavel Sklenář ***@***.***>:

…

Hi, I have developed a general approach for configuring Home Assistant, including the use of a reverse proxy. See #55 <#55> for details. @jlpedrosa <https://github.com/jlpedrosa> or others, it's quite a significant change, so it would be fine to test it before merging. Can you help me with this? I'm particularly concerned about using securityContext at any level. Many thanks. — Reply to this email directly, view it on GitHub <#30 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAASHYRTSUXQNQDOC7RBMQTZCHAHLAVCNFSM6AAAAABDCEZDPCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMBZGUZTCOJRG4> . You are receiving this because you commented.Message ID: ***@***.***>

[robertwbl]

I was having an issue where my trusted proxies weren't getting added to the configuration.yaml, and then realised that of course it only happens if the value ingress.enable = true, which in my case it is not because I'm using Gateway API and an HTTPRoute instead of an "old fashioned" ingress. I'm not sure what support you'd like to consider for the Kubernetes Gateway API given that's the way things are going, and I had the fairly easy workaround of removing the line {{- if .Values.ingress.enabled }}, but I thought it was worth mentioning in case others want to go down the gateway route.

[jlpedrosa]

Hey team, did not have time to test it, IMO it felt a bit dangerous.....

[pajikos]

I was having an issue where my trusted proxies weren't getting added to the configuration.yaml, and then realised that of course it only happens if the value ingress.enable = true, which in my case it is not because I'm using Gateway API and an HTTPRoute instead of an "old fashioned" ingress. I'm not sure what support you'd like to consider for the Kubernetes Gateway API given that's the way things are going, and I had the fairly easy workaround of removing the line {{- if .Values.ingress.enabled }}, but I thought it was worth mentioning in case others want to go down the gateway route.

Ok, good point, I am thinking about adding configuration property externalIngress = "true|false" to be able to include trusted_proxies in a case of using custom reverse proxy