Treedata sort with incremental data

[sapnildessai]

Hi @ghiscoding ,

I have a requirement to update slickgrid data incrementally, However when doing so tree data sorting breaks and the reason being there are lot of duplicate entries in childrens array. Please find below screen

Here is the details requirement:-

1. Load first 500 records from the server
2. Fire next server query to fetch next 500 set of records
3. Append second set of records to previous received data

If the grid is provided data after entire data set is ready then everything works fine. Its only when i assign data incrementally this problem occurs

Could you please guide me correct direction here?

Thanks

[sapnildessai]

Ok, I did some debugging in the lib and found out the problem.

1. Dataset that I am using is a flat data set with parentId set for treeview.
2. unflattenParentChildArrayToTree function in the lib converts flat data into hierarchical data structure. This mutates the original dataset i have passed.
3. Now when i send incremental data, it is already mutated in previous iteration which has children property set, here with new data that just came in, duplicate children getting added in this else condition
   const p = all[item[parentPropName]];
   if (!(childrenPropName in p)) {
   p[childrenPropName] = [];
   }
   p[childrenPropName].push(item);

Workaround that i did is, remove all children property when incremental data comes in and then feed it to slick grid.

Possible solution:-
Before this line of code
p[childrenPropName].push(item);

we can put if condition to check if the item with given id is already present in p[childrenPropName] array if not then only push it else skip it.
let me know your thoughts on this and i can raise a PR for your review.
Thanks

[ghiscoding]

Yes the reason is because the lib does mutate the data and it is clearly written in the Wiki under Tree Data - Important Note and I have done it this way for performance reasons since the lib already keeps 2 datasets (1x flat, 1x hierarchical) I didn't want to keep a 3rd deep copy array (you can do that on your side though). If you do keep a deep copy of the original dataset, then you can possibly use invalidateHierarchicalDataset(items?: Array) from the Grid Service which is used when inserting data... actually have you tried the addItems from the same Grid Service? It uses the invalidate and does take care of the transformation for you.

gridService.addItems(newItems);

The Example 28 demo has a button to add a single item at a time with addItem() but you can just use the plural method addItems

So at the end, you might think that loading your data in multiple shot might be less expensive but that might turn more expensive because you'll ask the lib to do 2 transform (1x on a smaller dataset and a 2nd time with a much bigger dataset). The recent version of the code is still quite performant but again remember that it transforms, mutate and keep 2x datasets internally to get it all working so take informative decisions according to that.