You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am running tests over the same Kafka Streams topology.
I have different suites extending EmbeddedKafkaStreamsAllInOne.
Each suite uses different ZK and Kafka ports, application name and topic names (same names with appName prefixed).
Each suite contains several tests, which use withRunningKafka and custom consumers, to test different parts of a KafkaStreams application that has several output topics.
When running tests sequentially they're fine, but in parallel starts showing errors of non-existent topics, dangling threads trying to connect to ZK, etc.
I've updated to 1.1.1 to no avail. I've seen this issue has been raised before here, and some fixes have been merged. Are they in 1.1.1 already?
First failure, when executing two suites with appNames "customer-test" and "order-test":
[2018-08-02 13:20:41,446] INFO Create topic: customer-test-sessionz (com.divvit.dp.streams.testing.KafkaStreamsTestUtils$)
[2018-08-02 13:20:41,448] INFO Create topic: order-test-sessionz (com.divvit.dp.streams.testing.KafkaStreamsTestUtils$)
[2018-08-02 13:20:41,452] WARN Error registering AppInfo mbean (org.apache.kafka.common.utils.AppInfoParser)
javax.management.InstanceAlreadyExistsException: kafka.admin.client:type=app-info,id=embedded-kafka-admin-client
at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
at org.apache.kafka.common.utils.AppInfoParser.registerAppInfo(AppInfoParser.java:62)
at org.apache.kafka.clients.admin.KafkaAdminClient.<init>(KafkaAdminClient.java:380)
at org.apache.kafka.clients.admin.KafkaAdminClient.createInternal(KafkaAdminClient.java:331)
at org.apache.kafka.clients.admin.AdminClient.create(AdminClient.java:62)
at net.manub.embeddedkafka.EmbeddedKafkaSupport.createCustomTopic(EmbeddedKafka.scala:716)
I understand here "winner takes it all", in this case "customers" Suite seems to enter first, then "orders" Suite is trying to use the same Kafka but it can't. Is it the client ID: "id=embedded-kafka-admin-client" of any importance here? It is hardcoded here, so I suppose it shouldn't be a matter of conflict.
Then I can find these errors later which seem to show second Suite complaining about its topics:
java.lang.IllegalArgumentException: Assigned partition order-test-customerOrderIntermediate-2 for non-subscribed topic regex pattern; subscription pattern is customer-test-customerOrderIntermediate|customer-test-customerz|customer-test-eventz|customer-test-visitorz
at org.apache.kafka.clients.consumer.internals.SubscriptionState.assignFromSubscribed(SubscriptionState.java:187)
at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.onJoinComplete(ConsumerCoordinator.java:220)
at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.joinGroupIfNeeded(AbstractCoordinator.java:367)
at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup(AbstractCoordinator.java:316)
at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:290)
at org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:1149)
at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1115)
at org.apache.kafka.streams.processor.internals.StreamThread.pollRequests(StreamThread.java:831)
at org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:788)
at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:749)
at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:719)
The text was updated successfully, but these errors were encountered:
xmariachi
changed the title
SBT Parallel execution of tests: topics deleted
SBT Parallel execution of tests failing
Aug 1, 2018
I am running tests over the same Kafka Streams topology.
I have different suites extending EmbeddedKafkaStreamsAllInOne.
Each suite uses different ZK and Kafka ports, application name and topic names (same names with appName prefixed).
Each suite contains several tests, which use
withRunningKafka
and custom consumers, to test different parts of a KafkaStreams application that has several output topics.When running tests sequentially they're fine, but in parallel starts showing errors of non-existent topics, dangling threads trying to connect to ZK, etc.
I've updated to 1.1.1 to no avail. I've seen this issue has been raised before here, and some fixes have been merged. Are they in 1.1.1 already?
First failure, when executing two suites with appNames "customer-test" and "order-test":
I understand here "winner takes it all", in this case "customers" Suite seems to enter first, then "orders" Suite is trying to use the same Kafka but it can't. Is it the client ID: "id=embedded-kafka-admin-client" of any importance here? It is hardcoded here, so I suppose it shouldn't be a matter of conflict.
Then I can find these errors later which seem to show second Suite complaining about its topics:
The text was updated successfully, but these errors were encountered: