JSON Streaming write and NUL characters or 0x00 writes for random objects

[vkumart]

Hi,
We are facing this intermittent issue where on a multi-object serialization(pretty large - Think of this as say a database serialization to JSON), at random places and for random objects the file contains NUL(0x00) for some writes. the content before and after are fine.

Please note that this is highly intermittent and we are not able to root cause the issue. Following are more details

• We are using Kotlin with JDK 17.0 and Kotlin

 <jackson.version>2.18.2</jackson.version>
 <kotlin.version>1.9.22</kotlin.version>

• We are using Jackson streaming as follows

val mapper = new jacksonObjectMapper()
            .registerModule(ParameterNamesModule())
            .registerModule(JavaTimeModule())
            .disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS)
            .setVisibility(PropertyAccessor.FIELD, JsonAutoDetect.Visibility.NON_PRIVATE)
            .setVisibility(PropertyAccessor.CREATOR, JsonAutoDetect.Visibility.ANY)
 val file = new File("test.json")
 val generator = objectMapper.factory.createGenerator(file.outputStream())
// all the entities, we call
 generator.writeObject(entity) // These entities are Kotlin data classes.

We do have a parallel write use case where multiple threads write to same json generator - by synchronized at the writing function level.

Questions

1. Is the access pattern correct?
2. Is there any known issues of random NUL writes happening?
3. Concurrent writes are supported? If not @synchronized should serialize writes, I supopose.

[cowtowncoder]

Quick note: JsonGenerator is not designed to be thread-safe -- intent is that only single thread would use it at any given time. So access to it needs to be synchronized if it is from multiple threads.

[cowtowncoder]

Oh, on this:

// all the entities, we call
 generator.writeObject(entity) // The

while legal, it's much better to call ObjectMapper.writeValue(gen, value) -- less overhead, retains context. I don't think that changes the issue here, but I mention it as sort of best practice.

[vkumart]

Thanks @cowtowncoder for your reply.

We are ensuring no serial access with the synchronized method. We even tried a sequential execution and the failure is happening.

Here are some more details

• The issue is that NUL characters are written instead of actual chacters. The length of write is same and the write and before are also same, just for these actual characters NUL are written. So we are not missing any write to the file but the write is not valid characters, indicating some corruption in memory but the length is incremented properly.
• Suspecting the issue could be at https://github.com/FasterXML/jackson-core/blob/2.19/src/main/java/com/fasterxml/jackson/core/io/SerializedString.java#L197 reusing the types

Any help on this is greatly appreciated, this is a blocker for us

[vkumart]

@cowtowncoder is this a known issue and have you seen any NUL writes happening in the past?

[vkumart]

Its not just 2 as I mentioned, The NULs are across the property names, their values. Values in the above cases are
Strings and Enums
You can see from the screenshot that the NUL actually tries to serialize the next available object and after the value 17304, you get description, hidden etc(not shown in screenshot)., so all those from _id(line no: 17060) to field(line no: 17062) excluding :value(which for the next object is 17304 are replaced by NUL and character counts exactly matches.

[cowtowncoder]

Ok what I am hoping to figure out is if this might be related to 2 likely things:

1. SerializedString handling (possibly initialization)
2. Buffer boundaries (and buffer recycling)

But even if it seemed to be related to one or the other, I don't really much of an idea of what to do about either case.

[vkumart]

Also type of fieldLayout and _id and field is the CUSTOM1 type and the dataclass looks like this

data class CUSTOM`(val type: Type, val namespace: String, val entityType: String, val entityId: String) {
@JsonValue
    fun value(): String {
        return StringJoiner(DELIMITER)
                .add(PREFIX)
                .add(type.value)
                .add(namespace)
                .add(entityType)
                .add(entityId)
                .toString()
    }

// and a JsonCreator as well to deserialize

[vkumart]

Is there any other way we can disable reusing of SerializedString and directly write(in an unoptimized way) through the generator or something of that sort that we can try to see if we can avoid this issue?

[cowtowncoder]

No, there is no such setting at this point.