You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It's common to add two tasks that have different file outputs without explicitly trying to do so. TaskGraph will raise an exception if two tasks could overwrite the same file but it also has the behavior of treating a Task uniquely based on its input hash arguments. This leads to cases when doing batch 2D convolution where coincidentally two kernel rasters have the same input arguments even though the context is different. Here's what TaskGraph does in that case:
Accept first task
Note second task is the same except for output so waits for the first one to be done
First task completes and updates the TaskGraph database that the task is complete
Second task runs, sees that the expected output is not its expected output so it runs the second identical task.
Second task completes and updates the TaskGraph database that the task is complete.
On a second iteration the first Task in the database will not have a target path list entry for its output because it was overwritten by the second task, so it churns triggering a re-run of every task below this that uses those files.
This is a bug and I suspect an an early attempt at copying a result to avoid recomputation but now we have explicit flags for that. This also turned out to be an apparently bad idea since it causes more recomputation than it prevents. This issue is about removing that duplicate functionality avoidance and instead allowing tasks to be uniquely defined by their input AND output arguments.
The text was updated successfully, but these errors were encountered:
It's common to add two tasks that have different file outputs without explicitly trying to do so. TaskGraph will raise an exception if two tasks could overwrite the same file but it also has the behavior of treating a Task uniquely based on its input hash arguments. This leads to cases when doing batch 2D convolution where coincidentally two kernel rasters have the same input arguments even though the context is different. Here's what TaskGraph does in that case:
On a second iteration the first Task in the database will not have a target path list entry for its output because it was overwritten by the second task, so it churns triggering a re-run of every task below this that uses those files.
This is a bug and I suspect an an early attempt at copying a result to avoid recomputation but now we have explicit flags for that. This also turned out to be an apparently bad idea since it causes more recomputation than it prevents. This issue is about removing that duplicate functionality avoidance and instead allowing tasks to be uniquely defined by their input AND output arguments.
The text was updated successfully, but these errors were encountered: