Flexible Sync and Optional Changes

[jensyth]

Hello,

I have been running into behavior that appears to contradict the documentation stating that optional schema changes are supported without additional work on the client side. In almost all real world scenarios, an optional schema change causes a Bad Changeset (DOWNLOAD) sync error. I am able to reproduce this behavior using a very simple modified template app (updated to allow use of version 1.5.1 although can also be reproduced on previous versions).

Setup:
Template Android App: kotlin.todo.flex
Flexible Sync Enabled, No Drafts, No Development Mode, No Client Recovery, Default Queryable Fields
Kotlin Realm SDK 1.5.1 (also seen in 1.5.0 and 1.0.0 which is original version in the template app)
No changes to client data models

Repro Steps:

• Launch the app, create/login account, create some items
• Verify sync is working with some change to item summary, back out of the app and clear from recent apps
• On server side, update and deploy a schema change to add an OPTIONAL field AND ALSO add an entry to a MongoDB document for this new OPTIONAL field (add to one document, add to all, it doesn't matter)
• Launch app again and note the sync error in logs
  com.mongodb.app W/SYNC: Connection[1]: Session[1]: Failed to integrate downloaded changesets: Failed to apply received changeset: Update: No such field: 'tag' in class 'class_Item' (instruction target: Item[ObjectId{639b9780b3d9d4148999e463}].tag, version: 26, last_integrated_remote_version: 19, origin_file_ident: 12, timestamp: 251086255045)
  and further down
  io.realm.kotlin.mongodb.exceptions.SyncException: [Client][BadChangeset(112)] Bad changeset (DOWNLOAD). at io.realm.kotlin.mongodb.internal.RealmSyncUtilsKt.convertSyncErrorCode(RealmSyncUtils.kt:82) at io.realm.kotlin.mongodb.internal.RealmSyncUtilsKt.convertSyncError(RealmSyncUtils.kt:70) at io.realm.kotlin.mongodb.internal.SyncConfigurationImpl._init_$lambda$2(SyncConfigurationImpl.kt:269) at io.realm.kotlin.mongodb.internal.SyncConfigurationImpl.$r8$lambda$-dp5z_oWgAG92DQlv34Y9CxDkrU(Unknown Source:0) at io.realm.kotlin.mongodb.internal.SyncConfigurationImpl$$ExternalSyntheticLambda0.onSyncError(Unknown Source:6)

Expected Results:

As long as the client data model does not make use of or access the new optional field, sync should continue working without needing additional handling or an update of the client data model

Actual Results:

Any combination of syncs where document data is received before or at the same time as the schema change (again, for an OPTIONAL field) causes a sync error. The only scenario where this behaves as one would expect is if you apply the schema change first, allow it to sync to the device, and then later populate that new field in the document.

This seems to contradict the documentation and seems to fail at many common real world scenarios. Is there some configuration setting or realm initialization step I am overlooking? Separately, is there any way to recover from this programmatically? Days of searching have only hinted at possible options using the Java SDK, or if we were on Partition Sync, or require updating the client data model (which effectively makes every optional schema change a force update on the app). I should note that this does NOT trigger a ClientReset and so I cannot make use of the ClientResetRequiredException.executeClientReset.

Thank you for any assistance you can provide!

[cmelchior]

Hi @jensyth

Yes, that does sound weird. And you are correct, what you expect should be actual behavior. We will look into it.

[ianpward]

@jensyth Are you able to share with us your app-id or Cloud UI URL so we can take a look at the logs on server-side? You can send it to [email protected] if you don't want to share here

[jensyth]

Hi @ianpward , I've sent this via email

[bainfu]

Hey @ianpward, I'm not sure this problem is unique to the realm-kotlin library.

[ianpward]

yes we've identified the issue and are working on a fix now - it is a server-side only change needed.

[ianpward]

this fix should be live now

[jensyth]

Thanks Ian, just tested the scenarios I had previously and they are all working now!

[anton-plebanovich]

We are facing the same issue for partition-based sync https://www.mongodb.com/community/forums/t/old-client-error-please-help/182896/4

[CBAffleck]

Also still seeing this issue for partition-based sync ^