Support YAML in Consul

[radcortez]

From quarkusio/quarkus#19127

[gsmet]

@radcortez as per our discussions, the Config Consul extension has been moved here. I thought Terraform would drop the other repo but it didn't, I will remove it manually.

[radcortez]

I remember about it, but I saw the other repo and I thought it was still active.

[radcortez]

Thanks!

[rodrigorodrigues]

Hi @radcortez,

I'm trying to upgrade my Quarkus App to the latest version(2.13.1.Final) and seems the binding for yml is not working anymore as previous version(2.5.0.Final), would know you some hack to make that works on the latest version please? Following piece of code and logs

@ApplicationScoped
public class AppLifecycleBean {

    @ConfigProperty(name = "security.oauth2.resource.jwt.keyValue")
    String jwtValue;

logs

ERROR [io.qua.dep.dev.IsolatedDevModeMain] (Quarkus Terminal Reader) Failed to start quarkus: io.quarkus.dev.appstate.ApplicationStartException: java.lang.RuntimeException: Failed to start quarkus
at io.quarkus.dev.appstate.ApplicationStateNotification.waitForApplicationStart(ApplicationStateNotification.java:58)
at io.quarkus.runner.bootstrap.StartupActionImpl.runMainClass(StartupActionImpl.java:122)
at io.quarkus.deployment.dev.IsolatedDevModeMain.restartApp(IsolatedDevModeMain.java:222)
at io.quarkus.deployment.dev.IsolatedDevModeMain.restartCallback(IsolatedDevModeMain.java:203)
at io.quarkus.deployment.dev.RuntimeUpdatesProcessor.doScan(RuntimeUpdatesProcessor.java:537)
at io.quarkus.deployment.dev.IsolatedDevModeMain$1$1.accept(IsolatedDevModeMain.java:130)
at io.quarkus.deployment.dev.IsolatedDevModeMain$1$1.accept(IsolatedDevModeMain.java:119)
at io.quarkus.deployment.console.ConsoleStateManager$1.accept(ConsoleStateManager.java:56)
at io.quarkus.deployment.console.ConsoleStateManager$1.accept(ConsoleStateManager.java:46)
at io.quarkus.dev.console.BasicConsole$2.run(BasicConsole.java:83)
at java.base/java.lang.Thread.run(Thread.java:1534)
Caused by: java.lang.RuntimeException: Failed to start quarkus
at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
at io.quarkus.runtime.Application.start(Application.java:101)
at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:110)
at io.quarkus.runtime.Quarkus.run(Quarkus.java:69)
at io.quarkus.runtime.Quarkus.run(Quarkus.java:42)
at io.quarkus.runtime.Quarkus.run(Quarkus.java:122)
at com.microservice.quarkus.QuarkusApplication.main(QuarkusApplication.java:9)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
at java.base/java.lang.reflect.Method.invoke(Method.java:577)
at io.quarkus.runner.bootstrap.StartupActionImpl$1.run(StartupActionImpl.java:103)
... 1 more
Caused by: javax.enterprise.inject.spi.DeploymentException: io.quarkus.runtime.configuration.ConfigurationException: Failed to load config value of type class java.lang.String for: security.oauth2.resource.jwt.keyValue
at io.quarkus.arc.runtime.ConfigRecorder.validateConfigProperties(ConfigRecorder.java:70)
at io.quarkus.deployment.steps.ConfigBuildStep$validateConfigValues1665125174.deploy_0(Unknown Source)
at io.quarkus.deployment.steps.ConfigBuildStep$validateConfigValues1665125174.deploy(Unknown Source)
... 11 more
Suppressed: java.util.NoSuchElementException: SRCFG00014: The config property security.oauth2.resource.jwt.keyValue is required but it could not be found in any config source
at io.smallrye.config.SmallRyeConfig.convertValue(SmallRyeConfig.java:294)
at io.smallrye.config.inject.ConfigProducerUtil.getValue(ConfigProducerUtil.java:104)
at io.quarkus.arc.runtime.ConfigRecorder.validateConfigProperties(ConfigRecorder.java:60)
at io.quarkus.deployment.steps.ConfigBuildStep$validateConfigValues1665125174.deploy_0(Unknown Source)
at io.quarkus.deployment.steps.ConfigBuildStep$validateConfigValues1665125174.deploy(Unknown Source)
at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
at io.quarkus.runtime.Application.start(Application.java:101)
at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:110)
at io.quarkus.runtime.Quarkus.run(Quarkus.java:69)
at io.quarkus.runtime.Quarkus.run(Quarkus.java:42)
at io.quarkus.runtime.Quarkus.run(Quarkus.java:122)
at com.microservice.quarkus.QuarkusApplication.main(QuarkusApplication.java:9)
... 4 more
Caused by: io.quarkus.runtime.configuration.ConfigurationException: Failed to load config value of type class java.lang.String for: security.oauth2.resource.jwt.keyValue
... 14 more

yml

spring:
  profiles: quarkus

security:
  oauth2:
    resource:
      jwt:
        keyValue: |
          -----BEGIN PUBLIC KEY-----
          XxxXX
          -----END PUBLIC KEY-----

[radcortez]

I've tried it, and it works for me. Can you provide me with a reproducer? Thanks!

[gjrtimmer]

Can somebody please share what the fix is to get this working. Yaml and Consul Config do not seem to work currently.

[gjrtimmer]

I currently do not have enough knowledge of Quarkus Internals to provide a PR with the required changes to implement the YAML support for Consul.

However, I would like to explain why this feature is essential.
From an operational point of view, it is essential to have human-readable configuration management. Regarding Quarkus within a microservice environment, I would currently have to implement a spring boot config server.

As a solutions architect, I will not sacrifice the human-readable ease of operations for DevOps by removing that. Furthermore, a spring boot config server will introduce additional maintenance, complexity, compute resources, and a layer of possible failure. I already have a complete Consul cluster with a Key/Value store synced across its cluster, being entirely reliant.

I would ask the community, what is needed and what kind of pre-work can I provide without an in-depth knowledge of Quarkus to have the consul-config extension be extended to be able to read the YAML config.

[radcortez]

Yes, YAML is not supported with Consul (YAML is supported with application.yaml and the config-yaml extension).

Well, as you can see, this issue is opened for more than an year with little interest, so this was not in our priorities.

All the YAML support for Config is already avalable. I guess what we need is a way to determine the payload type coming from Consul (in properties or YAML format) and use the proper reader to convert that to a ConfigSource. Can you check if there is a way to determine the format without having to guess it from the payload format?