Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Keep scanning supported directories in parallel with copying files found in the first directory #12

Open
andreiled opened this issue Nov 24, 2023 · 0 comments

Comments

@andreiled
Copy link
Owner

With the way how downloadAllNewFiles calls downloadNewFilesFromDir in a loop, the Downloader will always first copy all new files from the first discovered supported directory before scanning the second supported directory for new files and so forth.

Need to find a way for downloadNewFilesFromDir to queue copying of new files instead of actually copying them in a loop.

Note: I do not want to 'simply' queue copying using Node's built-in workers pool (i.e. by removing the await for the this.fileCopier.copy call) since that would result in copying multiple files in parallel which I do not feel will bring any value since all files are to be read from the same memory card and to be written to the same disk (block device) and so parallel workers would just end up competing with each other for the same limited IO resources.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant