Bugfixes #99
Open
Bugfixes #99
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Currently if any config option starts with `topic.` prefix, the connector will attempt to match it against regular expression for TopicWiseConfigs. If it fails then whole connector fails to start. This is a problem when launching on Confluent Cloud which adds for example `topic.creation.default.max.message.bytes` config. It mistakenly gets through the first check because it starts with `topic.` and then trips up on regular expression. It seems there could be many more configs like this that pertain to Kafka topic but are not this connectors TopicWiseConfigs. Adding suffix check should be sufficient and it should not break existing configurations as long as those adhered to the existing documentation.
If user sets connector up in schemaless mode and forgets to create the table, the connector tasks will fail with NullPointerException. Added warning should make it more obvious as to what is the issue. I am not adding throw here just in case there exists a situation where table creation could lag behind. Connector should continue working normally after the table is created.
|
@Bouncheck , could you please specify what bug does it fix |
|
The bug is collision between config names. We have our own pattern for extra configs which wrongly matches some more general configs added by some Kafka platforms. There is more details inside commit message. |
dkropachev
reviewed
Aug 2, 2024
| @@ -62,6 +64,10 @@ public class ScyllaDbSinkConnectorConfig extends AbstractConfig { | |||
| private static final Pattern TOPIC_KS_TABLE_SETTING_PATTERN = | |||
| Pattern.compile("topic\\.([a-zA-Z0-9._-]+)\\.([^.]+|\"[\"]+\")\\.([^.]+|\"[\"]+\")\\.(mapping|consistencyLevel|ttlSeconds|deletesEnabled)$"); | |||
|
|
|||
| private static final String[] TOPIC_WISE_CONFIGS_VALID_SUFFIXES = {".mapping",".consistencyLevel",".ttlSeconds",".deletesEnabled"}; | |||
There was a problem hiding this comment.
Could you please add some comments that briefly explains what this const is
dkropachev
approved these changes
Aug 2, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
One bugfix for config parsing and one additional logging.