Skip to content
This repository has been archived by the owner on Jan 8, 2020. It is now read-only.

--concurrency option seems to cause non-deterministic I/O errors #190

Open
hkeeler opened this issue Mar 24, 2016 · 0 comments
Open

--concurrency option seems to cause non-deterministic I/O errors #190

hkeeler opened this issue Mar 24, 2016 · 0 comments

Comments

@hkeeler
Copy link
Member

hkeeler commented Mar 24, 2016

I have not been able to confirm this bug, but through a fair amount of observation, it seems errors crop up when --concurrency or -c is set to more that 1. They are generally I/O errors, such as FTP download and file unzipping, and the errors happen for different files and FTP sites on subsequent runs of the process, but at least one usually occurs on a given run of the loader. I have attempted a couple runs of the loader with -c 1, and no errors occurred.

Also of note, I've only seen this while running in a Docker container. I don't see how that would cause this, but seemed worth mentioning.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

1 participant