You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It appears that when delta packs are being created, the work that needs done if first computed, and then all the work is parallelized. This ends up causing the machine to spend lots of time idle prior to parallelizing work. Would it be faster to, as soon as work is found that needs done, that that work is spun off as a task with no more than x tasks spun off at once?
The text was updated successfully, but these errors were encountered:
gtkramer
changed the title
Spend less time waiting for delta pack creation
[performance]: Spend less time waiting for delta pack creation
Nov 12, 2019
The main bottleneck for delta pack generation is sorting the bundles, which is currently not parallelized, as we are using default sorting which Go provides. We can compare writing a concurrent sorting algorithm and check how much improvement in performance can be achieved.
It appears that when delta packs are being created, the work that needs done if first computed, and then all the work is parallelized. This ends up causing the machine to spend lots of time idle prior to parallelizing work. Would it be faster to, as soon as work is found that needs done, that that work is spun off as a task with no more than x tasks spun off at once?
The text was updated successfully, but these errors were encountered: