Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove recursion from LocalRuntime during execution of FLFlow steps #792

Closed

Conversation

KeertiX
Copy link
Contributor

@KeertiX KeertiX commented Apr 14, 2023

TITLE: Remove recursion from LocalRuntime during execution of FLFlow steps

SUMMARY OF CHANGES:
Currently FLFlow steps are executed in a nested manner. This PR attemppts to remove recursion from LocalRuntime,
following approach is followed:

  1. call start()
  2. call execute_task
    - Function is executed till flow ends and arguments to execute the next step are updated by the FLSpec.next() method
    - This method is now split into 3 parts based on type of steps being executed:
    • execute_foreach_task: For executing collaborator steps
    • execute_agg_task: For executing aggregator steps
    • execute_end_task: For executing the last step in the flow

VERIFICATION SUMMARY:

  • Changes verified in Jupyter notebook and python file, with ray and single process backend

NEXT STEPS:

  • Further optimization to avoid serialization of clones in every collaborator step

@theakshaypant
Copy link
Collaborator

@KeertiX can you please resolve the comments/conflicts and request for review again?

@scngupta-dsp
Copy link
Contributor

@theakshaypant: Changes in this PR are already integrated. They were included in PR#791 and subsequently committed via PR #910

@theakshaypant
Copy link
Collaborator

@theakshaypant: Changes in this PR are already integrated. They were included in PR#791 and subsequently committed via PR #910

Thanks @scngupta-dsp ! Closing this one in that case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants