Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: prevents exception when the pipeline contains multiple nested loops #8677

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

etirelli
Copy link

@etirelli etirelli commented Dec 29, 2024

The cycle detection/breaking algorithm (PipelineBase::_break_supported_cycles_in_graph()) was raising exceptions in some situations for pipelines that contained nested loops. This was happening because the algorithm was trying to remove the same edge multiple times from the graph.

Related Issues

Proposed Changes:

  • added a check in PipelineBase::_break_supported_cycles_in_graph() to prevent the same edge from being removed multiple times

How did you test it?

  • included unit test

Notes for the reviewer

The current PR fixes the reported problem. However, it was clear during my tests that the current algorithm to break cycles is not deterministic (due to the undeterministic order of the dictionaries/lists it iterates on, in the presence of multiple Variadic and/or Optional inputs). I don't have enough knowledge at the moment to assess if this is really a problem, but my hunch is that in some situations, where multiple nodes with Variadic and/or Optional inputs are present, the end result of the pipeline execution might become undeterministic as well. Since this was not the point/scope of this ticket, I did not try to change that, but it might warrant a separate ticket/investigation.

Checklist

  • I have read the contributors guidelines and the code of conduct
  • I have updated the related issue with new insights and changes
  • I added unit tests and updated the docstrings
  • I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
  • I documented my code
  • I ran pre-commit hooks and fixed any issue

@etirelli etirelli requested review from a team as code owners December 29, 2024 17:56
@etirelli etirelli requested review from dfokina and Amnah199 and removed request for a team December 29, 2024 17:56
@CLAassistant
Copy link

CLAassistant commented Dec 29, 2024

CLA assistant check
All committers have signed the CLA.

…ops, due to the cycle detection removing the same edge multiple times (ref deepset-ai#8657)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cycle detection removes same edge multiple times
2 participants