Automatically sync when connectivity resumes

[lucian1900]

Right now, a manual sync is required after connectivity is resumed. Dualstorage should keep track of unsynced models and help with syncing them later.

[divineslight]

I am also concerned, it is written that "By default it saves every model locally" then how will I be at peace that the changed models are always synced to server so that when user opens the app from somewhere else he gets the same state?

[lucian1900]

You can't be at peace, at least not at the moment :)

You can do a manual sync periodically, which is what I'm doing atm.

[divineslight]

Thanks for the straight answer, but you can just set an option like sync-interval after which the local store would automatically sync to the server or some other scheme. So that the syncing functionality is also contained :)

[lucian1900]

That may be what I'll do. I'll look into this more closely, perhaps next week.

[divineslight]

Thumbs up!

[kurtmilam]

A potential solution here is to create an 'envelope' object that goes around each locally-stored model. The envelope could look something like this: {model:{/* model atts here */}, isDirty:true, lastLocalSave:1327764661, lastRemoteSync:1323275840}

To save space, use timestamps for times, one letter attribute names and 1/0 in place of true/false: {m:{/* model attributes here */},d:1,l:1327764661234,r:1323275840378}
Working example as an object on my fork.

To save more space, make the envelope an array, rather than an object. This saves 4 characters per array item per localStorage item over using an object with named attributes. Format: [model, lastLocalSave, lastRemoteSync, isDirty] . Example (using the same values from the previous examples): [{/* model attributes here */},1327764661534,1323275840678,1]
Working example as array

One more space-saver. Switch to seconds instead of microtime, switch from lastRemoteSync to lastRemoteSyncOffset (setting to 0 now - logic for this would still need to be implemented) and junk the isDirty item. If lastRemoteSyncOffset === 0, we're clean; if it's > 0 , we're dirty; if it's < 0, we were able to successfully sync remotely, but were unable to save to localStorage; if it's null, we have a model that's only been saved locally - never synced remotely. Format: [model, lastLocalSave, lastRemoteSyncOffset] . With previous values: [{/* model attributes here */},1327764661,4488821]
Working example, trimmed way down

It will need some fleshing out - for instance, setting the localSave and remoteSyncOffset values dynamically.

I only did these in javascript. If you'd like, I'll build some version into the CoffeeScript file, compile the .js from there and send a pull request. I actually suggest a mesh of the first and last ones - use an object for better extensibility but include all of the other trimming down I did in the other two examples.

[divineslight]

kurtmilam you sir are doing great :)

[lucian1900]

@kurtmilam Pretty nice. I think I'd prefer keeping a separate record that maps between keys and such data, rather than wrapping everything.

[kurtmilam]

Also a good idea. I'll rework it this evening or tomorrow. Any specific ideas on the implementation?

By the way, are you concerned at all about storage limitations with localStorage? I understand 5MB is a hard limit, and have read that Chrome stores strings encoded UTF-16, effectively halving the limit. According to that linked thread, the Chrome team isn't interested in upping the limit, either.

I think it'd be smart to consider writing dualSync in such a way that it can be extensible to work with different types of local storage - HTML5 localStorage, sqlite, indexed DB, etc.

[lucian1900]

I was thinking of something simpler, like localStorage['__dirty__'] = ['/foo/bar/1', '/foo/bar/2']. When a model is changed, it gets added to the dirty list, when the server gets the update the model gets removed from the dirty list. This might have to be a bit more complex and also remember the method, but I'm not certain right now.

As for separate adapters, it should be possible to simply put in a different Store class. Atm we're not worried about going over quotas since we're moving around little data.

[thiagobc]

I created a sync this on my fork: https://github.com/thiagobc/Backbone.dualStorage/tree/sync_local_and_remote .

I started with the idea to create a _dirty and _destroyed localstorage items.

• all dualsync requests will go first to onlineSync
• if onlineSync succeeds, localsync will be called with its results
• if onlineSync fails, localsync will be called marking dirty (if create and update) or destroyed (if delete)
• if there's any dirty/destroyed data locally, read requests will only execute localsync
• if there's no dirty/destroyed data locally, read requests will execute onlineSync first
• onlineSync to read requests will delete all local data, and then write them all again.
  this is done because would be hard to sync data destroyed on servers but existing locally
• a '/url_dirty' item will be created on localstorage to hold the ids of dirty records
• a '/url_destroyed' item will be created on localstorage to hold the ids of destroyed records

thoughts?

[lucian1900]

This is again similar to what I had in mind, but haven't had time for.

Looks nice.

[nilbus]

For future reference http://www.html5rocks.com/en/mobile/workingoffthegrid