Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] KinesisSink with jsonFlatten doesn't handle SchemaType.BYTES correctly #24129

Closed
3 tasks done
cbornet opened this issue Mar 26, 2025 · 1 comment · Fixed by #24132
Closed
3 tasks done

[Bug] KinesisSink with jsonFlatten doesn't handle SchemaType.BYTES correctly #24129

cbornet opened this issue Mar 26, 2025 · 1 comment · Fixed by #24132
Assignees
Labels
type/bug The PR fixed a bug or issue reported a bug

Comments

@cbornet
Copy link
Contributor

cbornet commented Mar 26, 2025

Search before asking

  • I searched in the issues and found nothing similar.

Read release policy

  • I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.

Version

HEAD

Minimal reproduce step

Run the following test

    @Test
    public void testSerializeRecordToJsonExpandingValue() throws Exception {
        SchemaType schemaType = SchemaType.AVRO;
        RecordSchemaBuilder valueSchemaBuilder = org.apache.pulsar.client.api.schema.SchemaBuilder.record("value");
        valueSchemaBuilder.field("a").type(SchemaType.BYTES).optional().defaultValue(null);
        GenericSchema<GenericRecord> valueSchema = Schema.generic(valueSchemaBuilder.build(schemaType));

        GenericRecord valueGenericRecord = valueSchema.newRecordBuilder()
                .set("a", "10".getBytes(StandardCharsets.UTF_8))
                .build();

        Record<GenericObject> genericObjectRecord = new Record<>() {

            @Override
            public org.apache.pulsar.client.api.Schema getSchema() {
                return valueSchema;
            }

            @Override
            public GenericObject getValue() {
                return valueGenericRecord;
            }
        };

        ObjectMapper objectMapper = new ObjectMapper().setSerializationInclusion(JsonInclude.Include.NON_NULL);
        String json = Utils.serializeRecordToJsonExpandingValue(objectMapper, genericObjectRecord, true);

        assertEquals(json, "{\"payload.a\":\"MTA=\"}");
    }

What did you expect to see?

The test should pass.

What did you see instead?

The test fails because payload.a is null.

java.lang.AssertionError:
Expected :{"payload.a":"MTA="}
Actual   :{"payload.a":null}

Anything else?

The problem is in the flattening library json-flattener which doesn't handle Jackson's BinaryNode.
In JsonJacksonValue, isString should also return true when the underlying JsonNode isBinary is true.

Are you willing to submit a PR?

  • I'm willing to submit a PR!
@cbornet cbornet added the type/bug The PR fixed a bug or issue reported a bug label Mar 26, 2025
@cbornet cbornet self-assigned this Mar 26, 2025
@cbornet
Copy link
Contributor Author

cbornet commented Mar 26, 2025

PR to fix in json-flattener: wnameless/json-base#9

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug The PR fixed a bug or issue reported a bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant