Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for initializing private attributes without callable #974

Merged
merged 24 commits into from
Jun 21, 2024
Merged

Add support for initializing private attributes without callable #974

merged 24 commits into from
Jun 21, 2024

Conversation

ishant162
Copy link
Collaborator

@ishant162 ishant162 commented May 20, 2024 •
edited
Loading

Background
Currently, Workflow_Interface supports initialization of private attributes via a callable function, In case of ray backend If user directly specifies the private attributes then the Participants fails to initialize private attributes.

Change description

  1. Updated Workflow_Interface and Experimental Aggregator based workflow to handle both options for setting Participants private attributes .
  • Set private attributes using a callable
  • Instead of callable, Set private attributes directly.
  1. Updated Workflow_interface documentation.
  2. Updated Workflow_Interface_101_MNIST.ipynb and Experimental workspace 101_torch_cnn_mnist to initialize private attributes by directly specifying.
  3. Workspace Export: Supports workspace creation from jupyter notebook where private attributes are directly specified.
  4. Workflow_Interface_1001_Workspace_Creation_from_JupyterNotebook.ipynb: Now supports workspace creation where private attributes are directly specified.
  5. Bug fix: Fx workspace create using Jupyter-notebook results in FileNotFoundError.
  6. Bumped tensorflow version: Workflow_Interface_104_Keras_MNIST_with_GPU.ipynb, openfl-workspace/experimental/104_keras_mnist
  7. Updated Workflow_Interface_401_MNIST_Aggregator_Validation_Ray_Watermarking.ipynb

Modifications

  • docs/about/features_index/workflowinterface.rst
  • openfl/experimental/runtime/local_runtime.py
  • openfl/experimental/interface/participants.py
  • openfl/experimental/interface/cli/workspace.py
  • openfl/experimental/component/aggregator/aggregator.py
  • openfl/experimental/component/collaborator/collaborator.py
  • openfl/experimental/workspace_export/export.py
  • openfl/experimental/federated/plan/plan.py
  • openfl-tutorials/experimental/Workflow_Interface_101_MNIST.ipynb
  • openfl-tutorials/experimental/Workflow_Interface_1001_Workspace_Creation_from_JupyterNotebook.ipynb
  • openfl-tutorials/experimental/Workflow_Interface_104_Keras_MNIST_with_GPU.ipynb
  • openfl-tutorials/experimental/Workflow_Interface_401_MNIST_Aggregator_Validation_Ray_Watermarking.ipynb
  • openfl-workspace/experimental/101_torch_cnn_mnist
  • openfl-workspace/experimental/104_keras_mnist/requirements.txt

Verification

  • Added Testcases to test the functionality of initialization of private attributes via both options (Workflow Interface & Experimental Aggregator based workflow):
    - tests/github/experimental/testflow_privateattributes_initialization_with_both_options.py
    - tests/github/experimental/testflow_privateattributes_initialization_without_callable.py
    - tests/github/experimental/workspace/testcase_private_attributes_initialization_with_both_options
    - tests/github/experimental/workspace/testcase_private_attributes_initialization_without_callable

  • Verified all testcases from tests/github/experimental

  • Verified all testcases from tests/github/experimental/workspace

  • Regression:
    - openfl-workspace/experimental/ : All workspaces
    - openfl-tutorials/experimental:
    - Workflow_Interface_101_MNIST.ipynb
    - Workflow_Interface_102_Aggregator_Validation.ipynb,
    - Workflow_Interface_103_Cyclic_Institutional_Incremental_Learning.ipynb,
    - Workflow_Interface_301_MNIST_Watermarking.ipynb
    - Workflow_Interface_104_Keras_MNIST_with_GPU.ipynb
    - Workflow_Interface_401_FedProx_with_Synthetic_nonIID.ipynb
    - Workflow_Interface_401_MNIST_Aggregator_Validation_Ray_Watermarking.ipynb

ishant162 added 15 commits May 16, 2024 17:48
@ishant162
Updated private_attrs initialization & Added testcases to test the same
01b8a5c 
Signed-off-by: ishant162 <[email protected]>
@ishant162
Fixed typo
daeaa84 
Signed-off-by: ishant162 <[email protected]>
@ishant162
Updated Workflow_Interface_101_MNIST.ipynb & workflowinterface.rst
175883f 
Signed-off-by: ishant162 <[email protected]>
@ishant162
Incorporated Internal review comments
ec1a6ef 
Signed-off-by: ishant162 <[email protected]>
@ishant162
Merge branch 'securefederatedai:develop' into update_private_attr_init
4d86169
@ishant162
Incorporated Internal review comments
151efa6 
Signed-off-by: Ishant Thakare <[email protected]>
@ishant162
Merge branch 'securefederatedai:develop' into update_private_attr_init
e11bb20
@ishant162
Experimental Aggregator based workflow: Updated private_attrs initial…
874340e 
…ization and added testcases to test the same

Signed-off-by: Ishant Thakare <[email protected]>
@ishant162
Fixed lint errors
5049ba2 
Signed-off-by: Ishant Thakare <[email protected]>
@ishant162
Bug fix: fx workspace create
61b68eb 
Signed-off-by: Ishant Thakare <[email protected]>
@ishant162
Workspace Export: Supports initialization of private attributes directly
9b966e7 
Signed-off-by: Ishant Thakare <[email protected]>
@ishant162
Fixed lint errors
c968c3d 
Signed-off-by: Ishant Thakare <[email protected]>
@ishant162
Incorporated Internal review comments
3632eb6 
Signed-off-by: Ishant Thakare <[email protected]>
@ishant162
Updated 101_torch_cnn_mnist private_attrs
a920ecf 
Signed-off-by: Ishant Thakare <[email protected]>
@ishant162
Updated workflow_interface tutorials 1001, 104_keras, 401_MNIST
aa2692b 
Signed-off-by: Ishant Thakare <[email protected]>
@ishant162 ishant162 changed the title Local Runtime: Support initialization of private attributes directly Add support for initializing private attributes without callable May 31, 2024
@ishant162 ishant162 marked this pull request as ready for review May 31, 2024 11:40
@ishant162 ishant162 requested a review from psfoley May 31, 2024 11:40
@ParthMandaliya ParthMandaliya marked this pull request as draft June 4, 2024 12:04
@ishant162 ishant162 marked this pull request as ready for review June 5, 2024 13:02
@ParthMandaliya @psfoley
Update docs/about/features_index/workflowinterface.rst
aa6f790 
Review comments incorporated.

Co-authored-by: Patrick Foley <[email protected]>
Signed-off-by: Parth Mandaliya <[email protected]>
@ParthMandaliya ParthMandaliya force-pushed the update_private_attr_init branch from 14fc736 to aa6f790 Compare June 13, 2024 08:26
@ParthMandaliya @psfoley
Update docs/about/features_index/workflowinterface.rst
0930a5a 
Review comments incorporated.

Co-authored-by: Patrick Foley <[email protected]>
Signed-off-by: Parth Mandaliya <[email protected]>
@ParthMandaliya ParthMandaliya force-pushed the update_private_attr_init branch from 5bc29ae to 0930a5a Compare June 13, 2024 08:28
@ParthMandaliya @psfoley
Update docs/about/features_index/workflowinterface.rst
a6342c0 
Review comments incorporated.

Co-authored-by: Patrick Foley <[email protected]>
Signed-off-by: Parth Mandaliya <[email protected]>
@ParthMandaliya ParthMandaliya force-pushed the update_private_attr_init branch from e0b8551 to a6342c0 Compare June 15, 2024 11:35
@ParthMandaliya ParthMandaliya requested a review from psfoley June 15, 2024 11:37
@ParthMandaliya ParthMandaliya self-assigned this Jun 15, 2024
@psfoley psfoley merged commit 604b81c into securefederatedai:develop Jun 21, 2024
24 of 26 checks passed
This was referenced Jun 24, 2024
Closed
Closed
manuelhsantana pushed a commit that referenced this pull request Jul 10, 2024
@ishant162 @ParthMandaliya
@psfoley @manuelhsantana
Add support for initializing private attributes without callable (#974)
17bdc70 
* Updated private_attrs initialization & Added testcases to test the same

Signed-off-by: ishant162 <[email protected]>

* Fixed typo

Signed-off-by: ishant162 <[email protected]>

* Updated Workflow_Interface_101_MNIST.ipynb & workflowinterface.rst

Signed-off-by: ishant162 <[email protected]>

* Incorporated Internal review comments

Signed-off-by: ishant162 <[email protected]>

* Incorporated Internal review comments

Signed-off-by: Ishant Thakare <[email protected]>

* Experimental Aggregator based workflow: Updated private_attrs initialization and added testcases to test the same

Signed-off-by: Ishant Thakare <[email protected]>

* Fixed lint errors

Signed-off-by: Ishant Thakare <[email protected]>

* Bug fix: fx workspace create

Signed-off-by: Ishant Thakare <[email protected]>

* Workspace Export: Supports initialization of private attributes directly

Signed-off-by: Ishant Thakare <[email protected]>

* Fixed lint errors

Signed-off-by: Ishant Thakare <[email protected]>

* Incorporated Internal review comments

Signed-off-by: Ishant Thakare <[email protected]>

* Updated 101_torch_cnn_mnist private_attrs

Signed-off-by: Ishant Thakare <[email protected]>

* Updated workflow_interface tutorials 1001, 104_keras, 401_MNIST

Signed-off-by: Ishant Thakare <[email protected]>

* Incorporated Internal review comments

Signed-off-by: Ishant Thakare <[email protected]>

* Resolving merge conflicts

Signed-off-by: Ishant Thakare <[email protected]>

* Updated import

Signed-off-by: Ishant Thakare <[email protected]>

* Fixed typo & updated federated plan

Signed-off-by: Ishant Thakare <[email protected]>

* Updated aggregator import

Signed-off-by: Ishant Thakare <[email protected]>

* Update docs/about/features_index/workflowinterface.rst

Review comments incorporated.

Co-authored-by: Patrick Foley <[email protected]>
Signed-off-by: Parth Mandaliya <[email protected]>

* Update docs/about/features_index/workflowinterface.rst

Review comments incorporated.

Co-authored-by: Patrick Foley <[email protected]>
Signed-off-by: Parth Mandaliya <[email protected]>

* Update docs/about/features_index/workflowinterface.rst

Review comments incorporated.

Co-authored-by: Patrick Foley <[email protected]>
Signed-off-by: Parth Mandaliya <[email protected]>

---------

Signed-off-by: ishant162 <[email protected]>
Signed-off-by: Ishant Thakare <[email protected]>
Signed-off-by: Parth Mandaliya <[email protected]>
Co-authored-by: ParthMandaliya <[email protected]>
Co-authored-by: Patrick Foley <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@psfoley psfoley psfoley approved these changes

Assignees

@ParthMandaliya ParthMandaliya

Labels
workflow_interface
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ishant162 @psfoley @ParthMandaliya