You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Consider the directory traversal code using parConcatIterate, the workers produce Left dirs and Right files. The workers need not produce Left dirs and can continue consuming those themselves unless there is a need for parallelism. Workers need to yield Left when there are idle workers (CPUs are not fully utilized). Somehow we need to make this feedback available to the workers.
One way is for the workers to periodically check the CPU utilization and yield more work for others to do if it is low and less if it is more. Another way may be to count active workers (not blocked for IO) and if active workers are less than a threshold then yield more work. But this may not work perfectly because even if there are workers idle CPUs might be busy doing some other work.
The text was updated successfully, but these errors were encountered:
Consider the directory traversal code using parConcatIterate, the workers produce
Left dirs
andRight files
. The workers need not produceLeft dirs
and can continue consuming those themselves unless there is a need for parallelism. Workers need to yieldLeft
when there are idle workers (CPUs are not fully utilized). Somehow we need to make this feedback available to the workers.One way is for the workers to periodically check the CPU utilization and yield more work for others to do if it is low and less if it is more. Another way may be to count active workers (not blocked for IO) and if active workers are less than a threshold then yield more work. But this may not work perfectly because even if there are workers idle CPUs might be busy doing some other work.
The text was updated successfully, but these errors were encountered: