Enable Connections and Policies to be configured by ConfigMaps

[mladBlum]

Setting up Eclipse Ditto fully automatically in a Kubernetes cluster using Helm is incredibly practical. However, to achieve a truly automated setup, there are some missing configuration options, Connections and Policies.

Currently, after successfully setting up Ditto, Connections and Policies need to be configured via API call. This requires an extra tool that monitors Ditto and executes the necessary API calls after Ditto has started.
Enabling the configuration of Connections and Policies within ConfigMaps would significantly simplify the deployment. Ditto could automatically read these resources and configure itself accordingly upon startup. This approach aligns with Kubernetes best practices and would streamline the deployment process.

I’m not on the development team, but here are a few thoughts. To implement such a feature, Ditto could be enhanced to process configurations provided by files in a designated folder, where ConfigMaps would be mounted.
This feature is probably also a benefit for classic server installations as well, allowing server administrators to pass the entire configuration at startup.

[thjaeckle]

Sure, that could be an additional way of configuring policies and connections.
This however has the downside that a restart of the pod is required to react on updated configmaps / files.
Maybe a file (e.g. via inode) watch would also work to apply config changes without restart.

The benefit of the current approach is that policies/connections can be dynamically added, adjusted, etc. - without any impact to all other policies/connections.
In my company's setup, we make use of a terraform-provider-restapi in order to manage policies and connection. Terraform (or Terragrunt in our case) "checks" the resources via Ditto's HTTP API and updates them in-place (if necessary), creates or deletes them, if needed.

If you want to contribute that in a way which does not destroy the current dynamic way, we surely will accept the contribution.
For our companies use-case I don't see much value, as we use connections and policies very dynamically - they are often changed, new ones come, old ones go, etc.