No way to reasonably customize core and horizon configuration files

[nyetwurk]

https://github.com/stellar/quickstart#customizing-configurations reads

It is recommended that you stop the container before editing any of these files, then restart the container after completing your customization.
NOTE: Be wary of editing these files. It is possible to break the services started within this container with a bad edit. It’s recommended that you learn about managing the operations of each of the services before customizing them, as you are taking responsibility for maintaining those services going forward.

We cannot deploy this way. We need a way to be able to reasonably configure every one of these files, but /start copy_defaults does not copy any of the critical files if their corresponding /etc dirs already exist (e.g. by providing a docker volume mount on the corresponding /etc directory). This makes it impossible to build custom configurations without a ton of work.

There should be a better way to merge the three following types of settings cleanly, and in the following order:

1. compile time defaults if no config file is specified (this appears to be a fatal error for core at least)
2. vendor provided configs
3. user configs

Many other programs provide various ways of doing this.

The ones that work the best are the ones where 2 and 3 are as minimal as possible, such that they only include the relevant differences from the default behavior.

The canonical way in many unix apps is by utilizing a config.d/ directory, which has a number of configuration files, possibly prefixed by an integer, e.g. 1- or 01- or 99- etc. such they can be parsed in a deterministic order.

Unfortunately, toml (let alone yaml/json) isn't very conducive to this approach.

Environment variables, however, are ideal for this as they are simple k/v pairs.

The hack of using sed to create toml files (from, say, a template) is one approach, but that requires that every possible user configurable option has a corresponding env var to go with it, which is probably impractical.

[nyetwurk]

In core's case, this might be done most easily by separating core configuration (which is key/value pairs) from quorum set configuration (which is a more complex set of structured data).

[nyetwurk]

Instead of quickstart, we are using this for soroban directly
https://github.com/stellar/soroban-tools/blob/main/cmd/soroban-rpc/docker/Dockerfile

[sreuland]

Hello, wanted to mention another alternative to consider in deployment, if it can leverage kubernetes, there are helm charts available, one for core, horizon, soroban-rpc here:

https://github.com/stellar/helm-charts/tree/main/charts

if kubernetes not an option, the manifests in those charts may still be good references for showing how to run the docker containers and the configurations.

[nyetwurk]

Hello, wanted to mention another alternative to consider in deployment, if it can leverage kubernetes, there are helm charts available, one for core, horizon, soroban-rpc here:

https://github.com/stellar/helm-charts/tree/main/charts

if kubernetes not an option, the manifests in those charts may still be good references for showing how to run the docker containers and the configurations.

Absolutely; i used exactly that to do our own configs.

[nyetwurk]

I don't think quickstart is suitable for production environments.

[mollykarcher]

I don't think quickstart is suitable for production environments.

Yes, we do not recommend it for production environments, which is why issues like this tend to fall in priority and not get a lot of engagement. That said, I think we could do more to make that explicit/clear in the README here and in our API documentation. And I realize that some partners were explicitly pointed to quickstart for soroban-rpc, given the timelines and lack of other available options yet, so I apologize for that!

[github-actions]

This issue is stale because it has been open for 30 days with no activity. It will be closed in 30 days unless the stale label is removed.

[leighmcculloch]

In #487 we removed the instructions for customizing the image. The instructions weren't consistent with what was actually possible. The image isn't maintained in such a way that customizing is really practical. Changes to the images routinely break any customization. Very likely custom setups would benefit from setups such as what is being used here: #469 (comment).

Please feel free to comment with alternatives if any occur. I'm going to close it since this aspect of the image is likely not to be improved.