[BUG Fix] Launching dependent LocalPipelineExecutors with skip_completed=False lead to waiting

[silverriver]

When launching dependent LocalPipelineExecutor, using the flag skip_completed=False in previous executor will lead to the following exector wait forever.

For example:

executor1 = LocalPipelineExecutor(
    pipeline=[
            ...
        ],
    tasks=10,
    logging_dir=f"logs/tokz",
    skip_completed=False
)

executor2 = LocalPipelineExecutor(
    pipeline=[
            ...
        ],
    tasks=10,
    logging_dir=f"logs/tokz",
)

if __name__ == "__main__":
    executor2.run()

The above code snippet will lead to

datatrove.executor.local:run:102 - Dependency job still has 10/10 tasks. Waiting...

even if executor1 has finished all its jobs.

[silverriver]

at @hynky1999 to get more attention

[guipenedo]

Thanks for the PR!
The idea of skip_completed is if you want to re-run the already completed tasks, so I believe the behavior from the second block is working as expected. If you don't want it to wait at all then just launch without the dependency

Also, no need to ping maintainers, we look into PRs and issues when we have time

[silverriver]

The second block will wait forever.

I think the correct/intended behavior of skip_completed is that: if we set skip_completed=False for the first block, then the first block will be re-runned and then the second block will be launched after the re-run is finished.

However, the current behavior is that: if we set skip_completed=False for the first block, then the first block will be re-runned, but the second block will be stucked in a waiting loop and never be launched. The reason is that executor1 .get_incomplete_ranks() will always return 10 tasks whenever it is called.

Thanks for the PR! The idea of skip_completed is if you want to re-run the already completed tasks, so I believe the behavior from the second block is working as expected. If you don't want it to wait at all then just launch without the dependency

Also, no need to ping maintainers, we look into PRs and issues when we have time