Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spend less time waiting for delta pack creation #645

Open
gtkramer opened this issue Nov 12, 2019 · 1 comment
Open

Spend less time waiting for delta pack creation #645

gtkramer opened this issue Nov 12, 2019 · 1 comment

Comments

@gtkramer
Copy link

It appears that when delta packs are being created, the work that needs done if first computed, and then all the work is parallelized. This ends up causing the machine to spend lots of time idle prior to parallelizing work. Would it be faster to, as soon as work is found that needs done, that that work is spun off as a task with no more than x tasks spun off at once?

@gtkramer gtkramer changed the title Spend less time waiting for delta pack creation [performance]: Spend less time waiting for delta pack creation Nov 12, 2019
@reaganlo reaganlo changed the title [performance]: Spend less time waiting for delta pack creation Spend less time waiting for delta pack creation Nov 13, 2019
@ashleshaAtrey ashleshaAtrey self-assigned this Mar 5, 2020
@ashleshaAtrey
Copy link
Contributor

The main bottleneck for delta pack generation is sorting the bundles, which is currently not parallelized, as we are using default sorting which Go provides. We can compare writing a concurrent sorting algorithm and check how much improvement in performance can be achieved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants