Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Avro deserialization fails in 1.1.18 #326

Open
kothapet opened this issue Feb 8, 2024 · 3 comments
Open

Avro deserialization fails in 1.1.18 #326

kothapet opened this issue Feb 8, 2024 · 3 comments

Comments

@kothapet
Copy link

kothapet commented Feb 8, 2024

I am trying to consume a topic written with Spring Cloud stream with kafka native serialization. We used GSR 1.1.17 version to write the messages.
My consumer also a spring cloud stream application with following configuration.

# for myConsumer function use the following customization
#
spring.cloud.stream.bindings.myConsumer-in-0.binder=kafka
spring.cloud.stream.bindings.myConsumer-in-0.destination=some-topic
spring.cloud.stream.bindings.myConsumer-in-0.group=some-grp

# Consumer AVRO config
spring.cloud.stream.kafka.bindings.myConsumer-in-0.consumer.configuration.spring.json.trusted.packages=*
spring.cloud.stream.kafka.bindings.myConsumer-in-0.consumer.configuration.key.deserializer=com.amazonaws.services.schemaregistry.deserializers.GlueSchemaRegistryKafkaDeserializer
spring.cloud.stream.kafka.bindings.myConsumer-in-0.consumer.configuration.value.deserializer=com.amazonaws.services.schemaregistry.deserializers.GlueSchemaRegistryKafkaDeserializer
# Consumer Glue Schema Registry config
spring.cloud.stream.kafka.bindings.myConsumer-in-0.consumer.configuration.region=us-west-2
spring.cloud.stream.kafka.bindings.myConsumer-in-0.consumer.configuration.registry.name=my-gsr-registry
spring.cloud.stream.kafka.bindings.myConsumer-in-0.consumer.configuration.dataFormat=AVRO
spring.cloud.stream.kafka.bindings.myConsumer-in-0.consumer.configuration.avroRecordType=SPECIFIC_RECORD
spring.cloud.stream.kafka.bindings.myConsumer-in-0.consumer.configuration.schemaNameGenerationClass=MyCustomSchemaNamingClass
spring.cloud.stream.kafka.bindings.myConsumer-in-0.consumer.configuration.schemaAutoRegistrationEnabled=false
#

Is there something I am missing?

2024-02-08 13:18:41.982 ERROR 2039185 --- [           main] org.apache.kafka.common.utils.Utils      : Failed to close consumer key deserializer with type com.amazonaws.services.schemaregistry.deserializers.GlueSchemaRegistryKafkaDeserializer

java.lang.NullPointerException: null
        at com.amazonaws.services.schemaregistry.deserializers.GlueSchemaRegistryKafkaDeserializer.close(GlueSchemaRegistryKafkaDeserializer.java:129) ~[schema-registry-serde-1.1.18.jar!/:na]
        at org.apache.kafka.common.utils.Utils.closeQuietly(Utils.java:1014) ~[kafka-clients-3.1.2.jar!/:na]
        at org.apache.kafka.clients.consumer.KafkaConsumer.close(KafkaConsumer.java:2400) [kafka-clients-3.1.2.jar!/:na]
        at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:820) [kafka-clients-3.1.2.jar!/:na]
        at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:664) [kafka-clients-3.1.2.jar!/:na]
        at org.springframework.kafka.core.DefaultKafkaConsumerFactory.createRawConsumer(DefaultKafkaConsumerFactory.java:483) [spring-kafka-2.8.11.jar!/:2.8.11]
        at org.springframework.kafka.core.DefaultKafkaConsumerFactory.createKafkaConsumer(DefaultKafkaConsumerFactory.java:451) [spring-kafka-2.8.11.jar!/:2.8.11]
        at org.springframework.kafka.core.DefaultKafkaConsumerFactory.createConsumerWithAdjustedProperties(DefaultKafkaConsumerFactory.java:427) [spring-kafka-2.8.11.jar!/:2.8.11]
        at org.springframework.kafka.core.DefaultKafkaConsumerFactory.createKafkaConsumer(DefaultKafkaConsumerFactory.java:394) [spring-kafka-2.8.11.jar!/:2.8.11]
        at org.springframework.kafka.core.DefaultKafkaConsumerFactory.createConsumer(DefaultKafkaConsumerFactory.java:371) [spring-kafka-2.8.11.jar!/:2.8.11]
        at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.<init>(KafkaMessageListenerContainer.java:778) [spring-kafka-2.8.11.jar!/:2.8.11]
        at org.springframework.kafka.listener.KafkaMessageListenerContainer.doStart(KafkaMessageListenerContainer.java:352) [spring-kafka-2.8.11.jar!/:2.8.11]
        at org.springframework.kafka.listener.AbstractMessageListenerContainer.start(AbstractMessageListenerContainer.java:461) [spring-kafka-2.8.11.jar!/:2.8.11]
        at org.springframework.kafka.listener.ConcurrentMessageListenerContainer.doStart(ConcurrentMessageListenerContainer.java:226) [spring-kafka-2.8.11.jar!/:2.8.11]
        at org.springframework.kafka.listener.AbstractMessageListenerContainer.start(AbstractMessageListenerContainer.java:461) [spring-kafka-2.8.11.jar!/:2.8.11]
        at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter.doStart(KafkaMessageDrivenChannelAdapter.java:326) [spring-integration-kafka-5.5.16.jar!/:5.5.16]
        at org.springframework.integration.endpoint.AbstractEndpoint.start(AbstractEndpoint.java:158) [spring-integration-core-5.5.16.jar!/:5.5.16]
        at org.springframework.cloud.stream.binder.AbstractMessageChannelBinder.doBindConsumer(AbstractMessageChannelBinder.java:484) [spring-cloud-stream-3.2.9.jar!/:3.2.9]
        at org.springframework.cloud.stream.binder.AbstractMessageChannelBinder.doBindConsumer(AbstractMessageChannelBinder.java:98) [spring-cloud-stream-3.2.9.jar!/:3.2.9]
        at org.springframework.cloud.stream.binder.AbstractBinder.bindConsumer(AbstractBinder.java:143) [spring-cloud-stream-3.2.9.jar!/:3.2.9]
        at org.springframework.cloud.stream.binding.BindingService.doBindConsumer(BindingService.java:184) [spring-cloud-stream-3.2.9.jar!/:3.2.9]
        at org.springframework.cloud.stream.binding.BindingService.bindConsumer(BindingService.java:137) [spring-cloud-stream-3.2.9.jar!/:3.2.9]
        at org.springframework.cloud.stream.binding.AbstractBindableProxyFactory.createAndBindInputs(AbstractBindableProxyFactory.java:118) [spring-cloud-stream-3.2.9.jar!/:3.2.9]
        at org.springframework.cloud.stream.binding.InputBindingLifecycle.doStartWithBindable(InputBindingLifecycle.java:58) [spring-cloud-stream-3.2.9.jar!/:3.2.9]
        at java.util.LinkedHashMap$LinkedValues.forEach(LinkedHashMap.java:608) ~[na:1.8.0_392]
        at org.springframework.cloud.stream.binding.AbstractBindingLifecycle.start(AbstractBindingLifecycle.java:57) ~[spring-cloud-stream-3.2.9.jar!/:3.2.9]
        at org.springframework.cloud.stream.binding.InputBindingLifecycle.start(InputBindingLifecycle.java:34) [spring-cloud-stream-3.2.9.jar!/:3.2.9]
        at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:178) ~[spring-context-5.3.25.jar!/:5.3.25]
        at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:54) ~[spring-context-5.3.25.jar!/:5.3.25]
        at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:356) ~[spring-context-5.3.25.jar!/:5.3.25]
        at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0_392]
        at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:155) ~[spring-context-5.3.25.jar!/:5.3.25]
        at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:123) ~[spring-context-5.3.25.jar!/:5.3.25]
        at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:935) ~[spring-context-5.3.25.jar!/:5.3.25]
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:586) ~[spring-context-5.3.25.jar!/:5.3.25]
        at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:731) ~[spring-boot-2.7.9.jar!/:2.7.9]
        at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:408) ~[spring-boot-2.7.9.jar!/:2.7.9]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:307) ~[spring-boot-2.7.9.jar!/:2.7.9]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1303) ~[spring-boot-2.7.9.jar!/:2.7.9]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1292) ~[spring-boot-2.7.9.jar!/:2.7.9]
        at com.github.kothapet.ScsGsrCon1.ScsGsrCon1Application.main(ScsGsrCon1Application.java:10) ~[classes!/:0.0.1-SNAPSHOT]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_392]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_392]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_392]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_392]
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) ~[app1118.jar:0.0.1-SNAPSHOT]
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:108) ~[app1118.jar:0.0.1-SNAPSHOT]
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) ~[app1118.jar:0.0.1-SNAPSHOT]
        at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:65) ~[app1118.jar:0.0.1-SNAPSHOT]

2024-02-08 13:18:41.986  INFO 2039185 --- [           main] o.a.kafka.common.utils.AppInfoParser     : App info kafka.consumer for consumer-my-scsgsrcon1-grp-2 unregistered
2024-02-08 13:18:41.987 ERROR 2039185 --- [           main] o.s.cloud.stream.binding.BindingService  : Failed to create consumer binding; retrying in 30 seconds

org.springframework.cloud.stream.binder.BinderException: Exception thrown while starting consumer:
        at org.springframework.cloud.stream.binder.AbstractMessageChannelBinder.doBindConsumer(AbstractMessageChannelBinder.java:542) ~[spring-cloud-stream-3.2.9.jar!/:3.2.9]
        at org.springframework.cloud.stream.binder.AbstractMessageChannelBinder.doBindConsumer(AbstractMessageChannelBinder.java:98) ~[spring-cloud-stream-3.2.9.jar!/:3.2.9]
        at org.springframework.cloud.stream.binder.AbstractBinder.bindConsumer(AbstractBinder.java:143) ~[spring-cloud-stream-3.2.9.jar!/:3.2.9]
        at org.springframework.cloud.stream.binding.BindingService.doBindConsumer(BindingService.java:184) [spring-cloud-stream-3.2.9.jar!/:3.2.9]
        at org.springframework.cloud.stream.binding.BindingService.bindConsumer(BindingService.java:137) [spring-cloud-stream-3.2.9.jar!/:3.2.9]
        at org.springframework.cloud.stream.binding.AbstractBindableProxyFactory.createAndBindInputs(AbstractBindableProxyFactory.java:118) [spring-cloud-stream-3.2.9.jar!/:3.2.9]
        at org.springframework.cloud.stream.binding.InputBindingLifecycle.doStartWithBindable(InputBindingLifecycle.java:58) [spring-cloud-stream-3.2.9.jar!/:3.2.9]
        at java.util.LinkedHashMap$LinkedValues.forEach(LinkedHashMap.java:608) ~[na:1.8.0_392]
        at org.springframework.cloud.stream.binding.AbstractBindingLifecycle.start(AbstractBindingLifecycle.java:57) ~[spring-cloud-stream-3.2.9.jar!/:3.2.9]
        at org.springframework.cloud.stream.binding.InputBindingLifecycle.start(InputBindingLifecycle.java:34) [spring-cloud-stream-3.2.9.jar!/:3.2.9]
        at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:178) ~[spring-context-5.3.25.jar!/:5.3.25]
        at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:54) ~[spring-context-5.3.25.jar!/:5.3.25]
        at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:356) ~[spring-context-5.3.25.jar!/:5.3.25]
        at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0_392]
        at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:155) ~[spring-context-5.3.25.jar!/:5.3.25]
        at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:123) ~[spring-context-5.3.25.jar!/:5.3.25]
        at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:935) ~[spring-context-5.3.25.jar!/:5.3.25]
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:586) ~[spring-context-5.3.25.jar!/:5.3.25]
        at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:731) ~[spring-boot-2.7.9.jar!/:2.7.9]
        at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:408) ~[spring-boot-2.7.9.jar!/:2.7.9]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:307) ~[spring-boot-2.7.9.jar!/:2.7.9]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1303) ~[spring-boot-2.7.9.jar!/:2.7.9]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1292) ~[spring-boot-2.7.9.jar!/:2.7.9]
        at com.github.kothapet.ScsGsrCon1.ScsGsrCon1Application.main(ScsGsrCon1Application.java:10) ~[classes!/:0.0.1-SNAPSHOT]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_392]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_392]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_392]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_392]
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) ~[app1118.jar:0.0.1-SNAPSHOT]
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:108) ~[app1118.jar:0.0.1-SNAPSHOT]
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) ~[app1118.jar:0.0.1-SNAPSHOT]
        at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:65) ~[app1118.jar:0.0.1-SNAPSHOT]
Caused by: org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
        at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:823) ~[kafka-clients-3.1.2.jar!/:na]
        at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:664) ~[kafka-clients-3.1.2.jar!/:na]
        at org.springframework.kafka.core.DefaultKafkaConsumerFactory.createRawConsumer(DefaultKafkaConsumerFactory.java:483) ~[spring-kafka-2.8.11.jar!/:2.8.11]
        at org.springframework.kafka.core.DefaultKafkaConsumerFactory.createKafkaConsumer(DefaultKafkaConsumerFactory.java:451) ~[spring-kafka-2.8.11.jar!/:2.8.11]
        at org.springframework.kafka.core.DefaultKafkaConsumerFactory.createConsumerWithAdjustedProperties(DefaultKafkaConsumerFactory.java:427) ~[spring-kafka-2.8.11.jar!/:2.8.11]
        at org.springframework.kafka.core.DefaultKafkaConsumerFactory.createKafkaConsumer(DefaultKafkaConsumerFactory.java:394) ~[spring-kafka-2.8.11.jar!/:2.8.11]
        at org.springframework.kafka.core.DefaultKafkaConsumerFactory.createConsumer(DefaultKafkaConsumerFactory.java:371) ~[spring-kafka-2.8.11.jar!/:2.8.11]
        at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.<init>(KafkaMessageListenerContainer.java:778) ~[spring-kafka-2.8.11.jar!/:2.8.11]
        at org.springframework.kafka.listener.KafkaMessageListenerContainer.doStart(KafkaMessageListenerContainer.java:352) ~[spring-kafka-2.8.11.jar!/:2.8.11]
        at org.springframework.kafka.listener.AbstractMessageListenerContainer.start(AbstractMessageListenerContainer.java:461) ~[spring-kafka-2.8.11.jar!/:2.8.11]
        at org.springframework.kafka.listener.ConcurrentMessageListenerContainer.doStart(ConcurrentMessageListenerContainer.java:226) ~[spring-kafka-2.8.11.jar!/:2.8.11]
        at org.springframework.kafka.listener.AbstractMessageListenerContainer.start(AbstractMessageListenerContainer.java:461) ~[spring-kafka-2.8.11.jar!/:2.8.11]
        at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter.doStart(KafkaMessageDrivenChannelAdapter.java:326) ~[spring-integration-kafka-5.5.16.jar!/:5.5.16]
        at org.springframework.integration.endpoint.AbstractEndpoint.start(AbstractEndpoint.java:158) ~[spring-integration-core-5.5.16.jar!/:5.5.16]
        at org.springframework.cloud.stream.binder.AbstractMessageChannelBinder.doBindConsumer(AbstractMessageChannelBinder.java:484) ~[spring-cloud-stream-3.2.9.jar!/:3.2.9]
        ... 31 common frames omitted
Caused by: java.lang.NoSuchFieldError: IDENTITY_PROVIDERS
        at software.amazon.awssdk.services.glue.DefaultGlueBaseClientBuilder.finalizeServiceConfiguration(DefaultGlueBaseClientBuilder.java:76) ~[glue-2.22.12.jar!/:na]
        at software.amazon.awssdk.awscore.client.builder.AwsDefaultClientBuilder.finalizeChildConfiguration(AwsDefaultClientBuilder.java:182) ~[aws-core-2.20.121.jar!/:na]
        at software.amazon.awssdk.core.client.builder.SdkDefaultClientBuilder.syncClientConfiguration(SdkDefaultClientBuilder.java:180) ~[sdk-core-2.20.121.jar!/:na]
        at software.amazon.awssdk.services.glue.DefaultGlueClientBuilder.buildClient(DefaultGlueClientBuilder.java:37) ~[glue-2.22.12.jar!/:na]
        at software.amazon.awssdk.services.glue.DefaultGlueClientBuilder.buildClient(DefaultGlueClientBuilder.java:25) ~[glue-2.22.12.jar!/:na]
        at software.amazon.awssdk.core.client.builder.SdkDefaultClientBuilder.build(SdkDefaultClientBuilder.java:147) ~[sdk-core-2.20.121.jar!/:na]
        at com.amazonaws.services.schemaregistry.common.AWSSchemaRegistryClient.<init>(AWSSchemaRegistryClient.java:116) ~[schema-registry-common-1.1.18.jar!/:na]
        at com.amazonaws.services.schemaregistry.common.AWSSchemaRegistryClient.<init>(AWSSchemaRegistryClient.java:128) ~[schema-registry-common-1.1.18.jar!/:na]
        at com.amazonaws.services.schemaregistry.deserializers.GlueSchemaRegistryDeserializationFacade.<init>(GlueSchemaRegistryDeserializationFacade.java:105) ~[schema-registry-serde-1.1.18.jar!/:na]
        at com.amazonaws.services.schemaregistry.deserializers.GlueSchemaRegistryKafkaDeserializer.configure(GlueSchemaRegistryKafkaDeserializer.java:91) ~[schema-registry-serde-1.1.18.jar!/:na]
        at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:709) ~[kafka-clients-3.1.2.jar!/:na]
        ... 45 common frames omitted
@bjanischevsky
Copy link

bjanischevsky commented Feb 26, 2024 •
edited
Loading

I have the same error but when producing a message, also using GSR 1.1.18, abridged stack trace below. This is the only web page I could find with "ava.lang.NoSuchFieldError: IDENTITY_PROVIDERS" text in it. This sems to be very new.
....
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at com.amazonaws.services.lambda.runtime.api.client.EventHandlerLoader$StreamMethodRequestHandler.handleRequest(EventHandlerLoader.java:378)
at com.amazonaws.services.lambda.runtime.api.client.EventHandlerLoader$2.call(EventHandlerLoader.java:905)
at com.amazonaws.services.lambda.runtime.api.client.AWSLambda.startRuntime(AWSLambda.java:245)
at com.amazonaws.services.lambda.runtime.api.client.AWSLambda.startRuntime(AWSLambda.java:197)
at com.amazonaws.services.lambda.runtime.api.client.AWSLambda.main(AWSLambda.java:187)
Caused by: java.lang.NoSuchFieldError: IDENTITY_PROVIDERS
at software.amazon.awssdk.services.glue.DefaultGlueBaseClientBuilder.finalizeServiceConfiguration(DefaultGlueBaseClientBuilder.java:76)
at software.amazon.awssdk.awscore.client.builder.AwsDefaultClientBuilder.finalizeChildConfiguration(AwsDefaultClientBuilder.java:182)
at software.amazon.awssdk.core.client.builder.SdkDefaultClientBuilder.syncClientConfiguration(SdkDefaultClientBuilder.java:180)
at software.amazon.awssdk.services.glue.DefaultGlueClientBuilder.buildClient(DefaultGlueClientBuilder.java:37)
at software.amazon.awssdk.services.glue.DefaultGlueClientBuilder.buildClient(DefaultGlueClientBuilder.java:25)
at software.amazon.awssdk.core.client.builder.SdkDefaultClientBuilder.build(SdkDefaultClientBuilder.java:147)
at com.amazonaws.services.schemaregistry.common.AWSSchemaRegistryClient.(AWSSchemaRegistryClient.java:116)
at com.amazonaws.services.schemaregistry.serializers.GlueSchemaRegistrySerializationFacade.(GlueSchemaRegistrySerializationFacade.java:69)
at com.amazonaws.services.schemaregistry.serializers.GlueSchemaRegistrySerializationFacade$GlueSchemaRegistrySerializationFacadeBuilder.build(GlueSchemaRegistrySerializationFacade.java:48)
at com.amazonaws.services.schemaregistry.serializers.GlueSchemaRegistryKafkaSerializer.configure(GlueSchemaRegistryKafkaSerializer.java:96)
at org.apache.kafka.clients.producer.KafkaProducer.(KafkaProducer.java:375)
... 15 more

@bjanischevsky
Copy link

bjanischevsky commented Mar 18, 2024 •
edited
Loading

I solved the earlier issue by upgrading all AWS maven dependencies.

Here´s a list of the dependencies I used:

<dependency>
  <groupId>junit</groupId>
  <artifactId>junit</artifactId>
  <version>4.13.2</version>
  <scope>test</scope>
</dependency>

<dependency>
  <groupId>org.junit.jupiter</groupId>
  <artifactId>junit-jupiter</artifactId>
  <version>RELEASE</version>
  <scope>test</scope>
</dependency>

<dependency>
  <groupId>com.amazonaws</groupId>
  <artifactId>aws-lambda-java-core</artifactId>
  <version>1.2.3</version>
</dependency>

<dependency>
  <groupId>com.amazonaws</groupId>
  <artifactId>aws-lambda-java-events</artifactId>
  <version>3.11.4</version>
</dependency>

<dependency>
  <groupId>com.amazonaws</groupId>
  <artifactId>aws-lambda-java-log4j2</artifactId>
  <version>1.6.0</version>
</dependency>

<dependency>
  <groupId>software.amazon.msk</groupId>
  <artifactId>aws-msk-iam-auth</artifactId>
  <version>2.0.3</version>
</dependency>

<dependency>
  <groupId>software.amazon.glue</groupId>
  <artifactId>schema-registry-serde</artifactId>
  <version>1.1.19</version>
  <!-- as of Feb 2025, 1.1.18 is the latest version
       AWS documentation shos 1.1.8 as the library to be used
   -->
</dependency>

<dependency>
  <groupId>com.google.code.gson</groupId>
  <artifactId>gson</artifactId>
  <version>2.10.1</version>
</dependency>

<dependency>
  <groupId>org.apache.kafka</groupId>
  <artifactId>kafka-clients</artifactId>
  <!-- <version>7.0.1-ccs</version> -->
  <version>3.7.0</version>
</dependency>

<dependency>
  <groupId>io.confluent</groupId>
  <artifactId>kafka-avro-serializer</artifactId>
  <!-- For Confluent Platform 7.0.1 -->
  <!-- <version>7.4.0</version> -->
  <version>7.6.0</version>
</dependency>

<dependency>
  <groupId>org.slf4j</groupId>
  <artifactId>slf4j-api</artifactId>
  <version>2.0.12</version>
</dependency>

<dependency>
  <groupId>org.slf4j</groupId>
  <artifactId>slf4j-simple</artifactId>
  <version>2.0.12</version>
</dependency>

<dependency>
  <groupId>com.google.protobuf</groupId>
  <artifactId>protobuf-java</artifactId>
  <version>3.25.3</version>
</dependency>

@bjanischevsky
Copy link

Back to the same problem, out of a sudden and unannounced.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@kothapet @bjanischevsky