Syncing all dirty models

[AntSwift]

I have a scenario where I have a Collection of Jobs and when navigating into a job, I request a collection of services.

The URL of the services collection is …/services/12345 and such its offline store name is …/services/12345_dirty.

When the app comes back online and I want to sync everything that has changed. To do this I create dummy Jobs and Services collections and execute syncDirtyAndDestroyed.

This approach works fine for Jobs because there is no dynamic element to the URL but services sync never completes due to the URL parameter.

I see two solutions check for and parse any '_dirty' elements in localstorage, create each matching collection in turn the sync or maintain a reference to all the services collection that are created then iterate over them to perform the sync. Both options seem rather crude.

Is there currently support for global syncs in the fashion is described?

    // perform global sync.
    var that = this;
    var jobsSync = new app.collections.JobsCollection()).fetch({ 
        success: function (collection, resp, options) {
            collection.syncDirtyAndDestroyed();    
        }
    });

    var servicesSync = new app.collections.ServicesCollection().fetch({ 
        success: function (collection, resp, options) {
            collection.syncDirtyAndDestroyed();    
        }
    });

[nilbus]

I'm trying to understand your question. If you have a dynamic URL, then you can override the store name your collection uses by setting the storeName attribute on your collection. Does that help?

I'm not exactly sure what you mean by a global sync, but Dualsync does not support syncing in association/relation dependency order, if that's what you're asking. Like Backbone, it does not track associations/relations.

Feel free to clarify if I did not understand your question.

[AntSwift]

storeName may solve the issue, I will check this out.

What I mean by global sync is, at the moment I have in localstorage something similar to this:
.../services/job1_dirty: [guid1, guid2, guid3]
.../services/job2_dirty: [guid4]
.../services/job3_dirty: [guid5, guid6]

where job1 - 3 are the ID's of the jobs and guid1-6 are the temp IDs of the dirty service objects.

In this state, I would need to perform multiple sync's to clean up all the dirty objects. Once for each Job ID. Instead I wish to sync all the services in one hit.

I hope this helps.

[AntSwift]

Setting the store name does indeed solve the issue I was having. I now have a single services_dirty entry in localstorage which is synced correctly.

I did get caught out by the attr.id bug that has very recently been fixed but a quick upgrade sorted that out.

Thanks for your help!

[nilbus]

I understand what you mean now.

No, there is no global sync right now, though that seems like a good idea. I think the only feasible option would be to keep a registry of all collections that use should be synced. The implementation of this idea would need to be carefully considered, as there are several edge cases. Not all collections are intended to be synced.

I'm glad storeName got you what you were looking for and that I found that idAttribute issue the other day. :-)

[AntSwift]

I think sync of a particular storeName is a good idea. In my scenario being able to perform a sync on app start or cordova's online event are important.

This could be implemented quite simply as a helper method:

Backbone.syncDirtyAndDestroyed("services");

As for the scenario where sync is not desired, would a disableLocal property with a default of false be too crude a method for disabling local storage on a collection instance (or model instance)?

var col = new CustomCollection({ disableLocal: true });

A collection created in this way would kick back to the default Backbone.Sync and never touch local storage although this would require quite a bit of refactoring.

[nilbus]

I'll rename and reopen this issue to see if any others have any feedback.

[elad]

I'm not convinced this is a problem/missing feature in DualStorage, and (at least at the moment) I think it would be wrong to add a "global sync" option or expose the storeName.

At least in my code, which makes a pretty heavy use of DualStorage, I've never come across the need to create "dummy" objects and/or issue a global sync... Like @nilbus said, this should be very carefully considered and implemented only if a use case stems from a real issue and not the result of oversight in design.

[AntSwift]

Hi eladxxx. Let see if I can elaborate on my scenario further.

I have a fairly simple hybrid mobile app which consists of a list of jobs which the user can drill into the details of. The details screen contains a number of tabs, one of which is services. Services are lazy loaded when needed from the API and since backbone is non-relational, are accessed with a separate ServicesCollection.

The user is free to make changes to services while offline and these are recorded and the services_dirty key is updated. This services collection is created as necessary when the user loads the services tab and I destroyed when the details view is removed (navigates back to the main job list for example).

Online and offline events are handled globally within the app. When the user is back online I need to sync any dirty object back to the server. Without a services collection, how do I do this?

I don’t get the point about exposing storeName, it is exposed and listed in the readme.

All told, my solution works. If there is a more elegant way of designing the app to mitigate the need to use dummy collections but keep a maintainable application structure then I'm all ears, clearly it is escaping me right now.

[elad]

Correct me if I'm wrong, but what you want to do is synchronize data that is in localStorage (saved by DualStorage while the app was offline) that you no longer have a reference to via a collection instance?

[AntSwift]

Yes, that's right. Since I can only call syncDirtyAndDestroyed on a collection or model (right?) I am rebuilding the collection from localStorage before performing the sync.

[elad]

Why can't you examine options.dirty in the success callback and if it's true keep a reference to it for when you regain connectivity, and then synchronize?

[AntSwift]

Yes sure I could maintain the dirty collections and models. However since dirty objects are already being tracked in localStorage, its not the most elegant solution.

Additionally, should there be an app crash, power cut, etc., these references would be lost anyway and i would need to fail back into localStorage to prevent data loss.

[elad]

Under the current design, dirty objects are only being tracked in localStorage when they are attached to a collection instance. They are read from localStorage only when a new Store object is instantiated, which can only happen when you operate on a collection instance. Furthermore, your assertion that it's not an elegant solution is incorrect, since this is the model by which all, or at least most, backing stores work, for example, virtual memory systems that use disks for swap. You always reference an object specifically (in virtual memory this is the virtual address), you never make blanket statements.

That said, I certainly see the need - the app crash use case is a compelling one which DualStorage must be able to cope with.

What worries me the most is that the proposed API can be easily abused: it does something that is already implemented, only in a potentially much broader fashion, and it operates behind the developer's back, since there's never a collection instance accessible to the developer. Also see my comment (#35 (comment)) regarding dirty object synchronization when two app instances run at the same time, and keep in mind that DualStorage still has no way of telling you when synchronization is finished... I feel with the proposed API things will only get more complicated, and I would really like to clean up the existing API before adding to it.

I think at the moment, I would at most accept an API that allows the developer to query DualStorage as to what collections are dirty, but still require the developer to create a collection instance and synchronize it explicitly.

[AntSwift]

We're arguing over semantics here. I still assert that reference tracking is not the right solution but the meat of my suggestion is the need to handle volatile scenarios, at least we have some common ground on this.

As you say, it is possible to do what I need using a globalSync method local to my app - I have a solution which works albeit a slightly clunky one.

[elad]

This isn't really a semantic issue because the proposed API is read/write and I think at most a read-only would be acceptable. A global API call also makes it impossible to work around the shared nature of localStorage, which I also think is unacceptable at least until DualStorage itself supports locking.

By the way, if you post your code here there's a good chance at least some of this functionality will make its way upstream, because the problem you're trying to solve is definitely important.