Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate Native API #1143

Closed
wants to merge 206 commits into from
Closed
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
206 commits
Select commit Hold shift + click to select a range
c23d7e8
Download src only for packages that exist on pip
MasterSkepticista Nov 7, 2024
ec25130
Install Gramine in base image
MasterSkepticista Nov 8, 2024
b70e4e1
Install Gramine in base image
MasterSkepticista Nov 8, 2024
6ca8e61
Python API code removed
tanwarsh Nov 12, 2024
6964401
Python API test code removed
tanwarsh Nov 12, 2024
3f273b7
Merge branch '12-Nov-24' of https://github.com/tanwarsh/openfl into 1…
tanwarsh Nov 12, 2024
b3297c0
Merge branch 'develop' into dockerize_gramine
MasterSkepticista Nov 12, 2024
004e99f
Add enclave template
MasterSkepticista Nov 12, 2024
1965115
SGX option in dockerize
MasterSkepticista Nov 12, 2024
e348058
Merge branch 'securefederatedai:develop' into 12-Nov-24
tanwarsh Nov 13, 2024
297802c
python api tutorial removed
tanwarsh Nov 13, 2024
190bf7e
openfl native removed from setup
tanwarsh Nov 13, 2024
f8845d8
delete Private Key Infrastructure workflow
tanwarsh Nov 13, 2024
7b90873
Documentation changes
tanwarsh Nov 13, 2024
8bafc05
Remove /keys
MasterSkepticista Nov 13, 2024
9be298c
Remove /host_save_path
MasterSkepticista Nov 13, 2024
382151f
Simplify variables and constants
MasterSkepticista Nov 13, 2024
f25ef3d
Handling for non-TLS scenarios
noopurintel Nov 13, 2024
a56f8f6
Log correction
noopurintel Nov 13, 2024
d8e4578
Merge branch 'develop' into develop
noopurintel Nov 13, 2024
547d496
deprecate interactive API
vrancurel Nov 12, 2024
6044e30
Renamed the workflow
noopurintel Nov 14, 2024
64c656c
removed python native API from documentation
tanwarsh Nov 14, 2024
b5e8bad
removed python native api logginf from tourch lim herovod
tanwarsh Nov 14, 2024
8a4cb51
Merge branch 'securefederatedai:develop' into 12-Nov-24
tanwarsh Nov 14, 2024
2aa9394
Use model name from test function name
noopurintel Nov 14, 2024
1b5c513
Remove testing code
noopurintel Nov 14, 2024
e0e95f9
Cleanup manifest and makefile; less hardcode
MasterSkepticista Nov 14, 2024
fe91455
Make venv default
MasterSkepticista Nov 14, 2024
80e34ec
Remove --sgx-ready flag
MasterSkepticista Nov 14, 2024
b773f9b
Job name change
noopurintel Nov 14, 2024
5952ef0
Store enclave measurement in file
MasterSkepticista Nov 14, 2024
f76875e
Rename dockerization.yml to tr_docker_native.yml
MasterSkepticista Nov 14, 2024
31c209a
Add TaskRunner Gramine-Direct CI E2E test
MasterSkepticista Nov 14, 2024
ff1619c
Update command help
MasterSkepticista Nov 14, 2024
12756b1
Rename tests
MasterSkepticista Nov 14, 2024
d5f3f0a
Merge branch 'develop' into dockerize_gramine
MasterSkepticista Nov 14, 2024
e4139ae
Add README.md
MasterSkepticista Nov 14, 2024
6cb5d05
Single workflow for TLS and non TLS both
noopurintel Nov 14, 2024
ca00053
Single workflow for TLS and non TLS both
noopurintel Nov 14, 2024
f15e9c5
Single workflow for TLS and non TLS both
noopurintel Nov 14, 2024
0591cc2
Non TLS with single model and 3.10 python
noopurintel Nov 14, 2024
5c3ab73
Merge pull request #1148 from noopurintel/develop
teoparvanov Nov 14, 2024
63177f4
Merge pull request #1125 from vrancurel/deprecate_interactive_api
teoparvanov Nov 14, 2024
9d9a624
Merge branch 'develop' into dockerize_gramine
MasterSkepticista Nov 15, 2024
1d7c9bf
Merge pull request #1140 from MasterSkepticista/dockerize_gramine
MasterSkepticista Nov 15, 2024
a977fb3
Fix compression pipeline hardcode for `fx model save` (#1152)
kta-intel Nov 16, 2024
82270de
Updates to straggler handling functionality (#996)
ParthMandaliya Sep 23, 2024
2c883ee
build(deps): update pyzmq requirement from <=26.1.1 to <=26.2.0 (#1032)
dependabot[bot] Sep 24, 2024
6cfc747
build(deps): bump pytest-asyncio from 0.23.8 to 0.24.0 (#1031)
dependabot[bot] Sep 24, 2024
6d0ee8f
updating trivy latest version and adding logging
rajithkrishnegowda Sep 25, 2024
2ca2eb9
Mark `create(...)` externally callable. (#1062)
MasterSkepticista Sep 26, 2024
07a6b1b
Experimental Aggregator Workflow: Fixed private attributes issue and …
refai06 Sep 27, 2024
ae9ddb9
Experimental Aggregator Workflow: Fixed private attributes issue and …
refai06 Oct 1, 2024
af38731
Experimental Aggregator Workflow: Updated the code and optimzed
refai06 Oct 4, 2024
2762731
Experimental Aggregator Workflow: Updated the code and optimzed
refai06 Oct 4, 2024
50cb42f
enable re-installation of requirements on fx plan initialize
kta-intel Sep 27, 2024
94d944e
lint fix
kta-intel Sep 27, 2024
a1eafab
uploading hadolint CSV report
rajithkrishnegowda Sep 26, 2024
2314041
changes to json format
rajithkrishnegowda Sep 26, 2024
ba3691f
fix typo error
rajithkrishnegowda Sep 26, 2024
a1d6857
Fix GaNDLF CI (#1072)
psfoley Sep 30, 2024
838ef68
build(deps): bump pytest from 8.3.2 to 8.3.3 (#1063)
dependabot[bot] Sep 30, 2024
c6b687b
fix system call to be functional on windows
kta-intel Oct 2, 2024
80c681f
PyTorch template workspace
teoparvanov Sep 26, 2024
da2d989
Update ubuntu.yml (#1081)
psfoley Oct 3, 2024
3e2b35a
Dependabot: Scan only core dependencies (#1077)
theakshaypant Oct 3, 2024
be3a4e8
Update version to 1.6 (#1083)
psfoley Oct 4, 2024
e9f214e
- Fix docker permissions issue on src copy command to default user/group
ishaileshpant Oct 3, 2024
d940e3c
Experimental Aggregator Workflow: Comments update
refai06 Oct 7, 2024
7361196
Provide a standard way to get data_loader in util contexts (#1076)
jkk-intel Oct 7, 2024
bb13bbf
Update releases.md to v1.6 (#1085)
psfoley Oct 9, 2024
15ce1b2
adding trivy scan for code and uploading results to artifacts
rajithkrishnegowda Sep 26, 2024
fab3c54
trivy code vulnerablity fix
rajithkrishnegowda Sep 26, 2024
4cf83bb
adding step to install trivy
rajithkrishnegowda Sep 26, 2024
c0d9f48
changing it to json as csv is not supported
rajithkrishnegowda Sep 26, 2024
2d40166
change to trivy step
rajithkrishnegowda Sep 26, 2024
9f0035b
added format to spdx-json for SBOM
rajithkrishnegowda Sep 27, 2024
7d03295
adding spdxjson step
rajithkrishnegowda Sep 30, 2024
97e2610
changes
rajithkrishnegowda Sep 19, 2024
a130db2
adding autoflake to precommit check
rajithkrishnegowda Sep 19, 2024
88ad9c3
fixed changes requested in comments
rajithkrishnegowda Sep 20, 2024
a2fa597
changes for not removing pass command
rajithkrishnegowda Sep 20, 2024
5cd4a44
changes for not removing pass command in 2 files
rajithkrishnegowda Sep 20, 2024
00143d7
made comment changes
rajithkrishnegowda Sep 20, 2024
1eb24e8
fix precommit issue
rajithkrishnegowda Sep 20, 2024
ef00401
reverted noqa comments
rajithkrishnegowda Sep 20, 2024
888ff23
changed pre-commit url
rajithkrishnegowda Sep 20, 2024
128e17f
added changes from the comments
rajithkrishnegowda Sep 23, 2024
54f7cbd
pre-commit fix
rajithkrishnegowda Oct 4, 2024
8a84fd5
adding docker bench security tool
rajithkrishnegowda Sep 30, 2024
43bf1f5
changing to valid names
rajithkrishnegowda Sep 30, 2024
090ce17
fix name
rajithkrishnegowda Sep 30, 2024
1bfccb2
adding Build Docker Bench Security Image
rajithkrishnegowda Oct 1, 2024
3081bf6
remove pulling image
rajithkrishnegowda Oct 1, 2024
99586bc
Experimental Aggregator Workflow: code update and Localruntime:Partic…
refai06 Oct 16, 2024
1294ac2
build(deps): update protobuf requirement
dependabot[bot] Aug 27, 2024
015995a
comments update
refai06 Oct 17, 2024
dc3d491
added git
tanwarsh Oct 17, 2024
9b88fbb
added git
tanwarsh Oct 17, 2024
38ef1d5
Remove unused arg and docstring (#1090)
MasterSkepticista Oct 18, 2024
7c9864a
added no-browser (headless) option to run jupyter notebooks on VM. (#…
tanwarsh Oct 21, 2024
ffe9ca3
[openfl-docker] Remove startup script dependency (#1091)
MasterSkepticista Oct 22, 2024
6cced01
Fixing manual PKI setup commands in taskrunner.rst
teoparvanov Oct 21, 2024
74a2844
Logging fix - No of rounds saved vs started
noopurintel Oct 22, 2024
6f694ef
Adding Workflow_Interface_101_MNIST To CI Pipeline
noopurintel Oct 24, 2024
a60998f
Corrected the path
noopurintel Oct 24, 2024
764d05c
Run for multiple versions
noopurintel Oct 24, 2024
366db7b
Ubuntu 22.04 and python 3.10
noopurintel Oct 24, 2024
a8c3a16
3.10 as string
noopurintel Oct 24, 2024
3ca2b00
Enable Workflow Interface 101 MNIST For PR Pipeline
noopurintel Oct 24, 2024
c57ecd5
build(deps): bump onnx in /openfl-workspace/gandlf_seg_test
dependabot[bot] Oct 23, 2024
a35e190
Optimize code: attribute check
refai06 Oct 23, 2024
299ab27
Added ipywidgets (#1103)
tanwarsh Oct 25, 2024
9e45f57
fix filename coverity issue
rajithkrishnegowda Oct 24, 2024
15f3ccc
fix code format issue
rajithkrishnegowda Oct 24, 2024
6f22ab3
reverting code check format issue
rajithkrishnegowda Oct 24, 2024
793c1f5
fix coverity hash issue in metaflow_utils file
rajithkrishnegowda Oct 25, 2024
c61a840
dummy commit workflow_interface
rajithkrishnegowda Oct 25, 2024
93de4f8
trigger withworkflow_interface
rajithkrishnegowda Oct 25, 2024
6636817
Rename experimental interface notebooks
MasterSkepticista Oct 25, 2024
61ba9a5
Update CI test
MasterSkepticista Oct 25, 2024
79099b8
Refactor `dockerize` and `export` (#1096)
MasterSkepticista Oct 29, 2024
8d4b8ff
Eliminate package install during import (#1109)
MasterSkepticista Oct 29, 2024
0d63109
Update links post nb renames (#1111)
MasterSkepticista Oct 30, 2024
f3eaa42
Rename scan.yml to hadolint.yml
gbikkiintel Oct 30, 2024
7f213d2
Fix trivy issue
gbikkiintel Oct 30, 2024
bf8f895
Fixed trivy version
gbikkiintel Oct 30, 2024
dbf7bbb
Fix for tensor shape error in Federated_Pytorch_MNIST_Tutorial.ipynb …
tanwarsh Nov 4, 2024
b556fea
Fixes for flake8 linting
rajithkrishnegowda Nov 4, 2024
2f1220a
added comments
rajithkrishnegowda Nov 4, 2024
742ae92
revert max line length to 100
rajithkrishnegowda Nov 4, 2024
16280a5
Fixing flake8
grib0ed0v Nov 1, 2024
f97c606
Validating Log File Path for cli function
rajithkrishnegowda Nov 4, 2024
cf93bd6
fix pre-commit issue
rajithkrishnegowda Nov 4, 2024
505e640
Validating Log File Path for cli function
rajithkrishnegowda Nov 4, 2024
7dd162c
fix linting issue
rajithkrishnegowda Nov 4, 2024
edcef06
fix(workspaces): reading shard number from default path
theakshaypant Aug 26, 2024
03e608c
fix(workspaces): added valueerror handling where data path represents…
theakshaypant Aug 26, 2024
12e3975
fix(docs): removed optional from collaborator create command as omitt…
theakshaypant Aug 26, 2024
d1c99b4
fix(openfl-workspaces): added type check for data path as shard num (…
theakshaypant Aug 26, 2024
e2ac805
fix(openfl-workspaces): changed error message for ValueError
theakshaypant Aug 27, 2024
dd537f7
restructure: linter changes
theakshaypant Oct 17, 2024
5f21333
fix(task runner examples): add a more descriptive error message
theakshaypant Nov 4, 2024
89e6a5b
fix(task runner docs): revert to DATA_PATH being optional
theakshaypant Nov 4, 2024
e20790e
Purge openfl-gramine
MasterSkepticista Nov 5, 2024
6d61e9d
Remove test, subcommand and module install
MasterSkepticista Nov 5, 2024
9b2f101
Remove outdated gramine workspaces
MasterSkepticista Nov 5, 2024
6ea8f31
Wiring flake8 with the PR pipeline
teoparvanov Nov 8, 2024
43cf503
Fixing code formatting issues reported by flake8
teoparvanov Nov 8, 2024
c767074
Update ubuntu.yml to use custom lint configs as in lint.yml
teoparvanov Nov 8, 2024
dc38840
Revert "Fixing code formatting issues reported by flake8"
teoparvanov Nov 11, 2024
bc9d562
fix for error while loading state dict in cifar10
tanwarsh Nov 7, 2024
5521a75
fix for error while loading state dict in cifar10
tanwarsh Nov 7, 2024
6ddb63b
Updated pull request types and disabled triggering PRs on drafts
gbikkiintel Nov 8, 2024
94dec0d
Python API code removed
tanwarsh Nov 12, 2024
4efee6c
Python API test code removed
tanwarsh Nov 12, 2024
94bdcd0
Set FQDN as env variable for pytest coverage
noopurintel Nov 12, 2024
a8e92de
Moving env variable outside
noopurintel Nov 12, 2024
2a78b19
Pytest framework implementation with Task Runner GitHub Workflow
noopurintel Nov 8, 2024
ee55f7a
Logging fix
noopurintel Nov 8, 2024
d0bbcd1
Fix code format issue
noopurintel Nov 9, 2024
1f75ad0
Modified openfl_e2e to end_to_end along with other occurrences
noopurintel Nov 11, 2024
08dc522
Error handling in participants file
noopurintel Nov 11, 2024
96859f4
README.md update
noopurintel Nov 11, 2024
81ee2a4
Incorporated review comments
noopurintel Nov 11, 2024
6a185d7
Final changes after testing
noopurintel Nov 11, 2024
426cf9f
Sample tests file mentioned in README.md
noopurintel Nov 11, 2024
3f715de
Added lxml to test requirements file
noopurintel Nov 11, 2024
3e7fe9e
Removed __init__.py file and corrected certify functions
noopurintel Nov 11, 2024
5d7dbb6
Revert pull_request trigger
noopurintel Nov 11, 2024
d9de677
Review comments addressed
noopurintel Nov 12, 2024
e2dd11c
Modified
noopurintel Nov 12, 2024
627c122
Revert testing changes
noopurintel Nov 12, 2024
9a1e968
Use logging directly after conftest initialisation
noopurintel Nov 12, 2024
31ce02d
Removed pull_request trigger after testing
noopurintel Nov 12, 2024
b4a384b
[FeTS][GaNDLF] skip loading `train.csv` during inference mode (#1141)
kta-intel Nov 12, 2024
0cbc007
python api tutorial removed
tanwarsh Nov 13, 2024
b7af60b
openfl native removed from setup
tanwarsh Nov 13, 2024
160d8ff
delete Private Key Infrastructure workflow
tanwarsh Nov 13, 2024
fd7d81a
Documentation changes
tanwarsh Nov 13, 2024
2b27c03
removed python native API from documentation
tanwarsh Nov 14, 2024
7a3662c
removed python native api logginf from tourch lim herovod
tanwarsh Nov 14, 2024
d021c6e
Logic to print score and training details
noopurintel Nov 13, 2024
dd4244d
Add collab and rounds to summary
noopurintel Nov 13, 2024
fe8c14f
Use only agg accuracy
noopurintel Nov 13, 2024
2b656db
Typo fix
noopurintel Nov 13, 2024
311e9f3
Renamed summary file
noopurintel Nov 13, 2024
b66023a
Job name as tr to help in ZephyrScale later
noopurintel Nov 13, 2024
36cbafa
Remove comment related to ZS
noopurintel Nov 13, 2024
48c49f9
TESTING: DON"T MERGE
gbikkiintel Nov 13, 2024
94a837d
Added trivy db
gbikkiintel Nov 13, 2024
29327f4
Python API code removed
tanwarsh Nov 12, 2024
64595a8
Python API test code removed
tanwarsh Nov 12, 2024
ca2c237
python api tutorial removed
tanwarsh Nov 13, 2024
c4be680
openfl native removed from setup
tanwarsh Nov 13, 2024
02d8380
delete Private Key Infrastructure workflow
tanwarsh Nov 13, 2024
607d341
Documentation changes
tanwarsh Nov 13, 2024
a5490ef
removed python native API from documentation
tanwarsh Nov 14, 2024
b15c599
removed python native api logginf from tourch lim herovod
tanwarsh Nov 14, 2024
a8a25b7
Merge branch '12-Nov-24' of https://github.com/tanwarsh/openfl into 1…
tanwarsh Nov 18, 2024
2c56964
Merge branch '12-Nov-24' of https://github.com/tanwarsh/openfl into 1…
tanwarsh Nov 18, 2024
4f673d1
Merge branch '12-Nov-24' of https://github.com/tanwarsh/openfl into 1…
tanwarsh Nov 18, 2024
556fbcd
Merge branch '12-Nov-24' of https://github.com/tanwarsh/openfl into 1…
tanwarsh Nov 18, 2024
cec875a
Merge branch '12-Nov-24' of https://github.com/tanwarsh/openfl into 1…
tanwarsh Nov 18, 2024
a62350b
Revert "removed python native api logginf from tourch lim herovod"
tanwarsh Nov 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 0 additions & 52 deletions .github/workflows/pki.yml

This file was deleted.

17 changes: 0 additions & 17 deletions docs/about/features_index/pynative.rst

This file was deleted.

14 changes: 1 addition & 13 deletions docs/developer_guide/advanced_topics/overriding_agg_fn.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,19 +7,7 @@
Override Aggregation Function
*****************************

With the aggregator-based workflow, you can use custom aggregation functions for each task via Python\*\ API or command line interface.


Python API
==========

1. Create an implementation of :class:`openfl.interface.aggregation_functions.core.AggregationFunction`.

2. In the ``override_config`` keyword argument of the :func:`openfl.native.run_experiment` native function, pass the implementation as a ``tasks.{task_name}.aggregation_type`` parameter.

.. note::
See `Federated PyTorch MNIST Tutorial <https://github.com/intel/openfl/blob/develop/openfl-tutorials/Federated_Pytorch_MNIST_custom_aggregation_Tutorial.ipynb>`_ for an example of the custom aggregation function.

With the aggregator-based workflow, you can use custom aggregation functions for each task via command line interface.

Command Line Interface
======================
Expand Down

This file was deleted.

2 changes: 1 addition & 1 deletion docs/developer_guide/running_the_federation.notebook.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
.. _running_notebook:

**********************************
Aggregator-Based Workflow Tutorial
Aggregator-Based Workflow Tutorial (Deprecated)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't related to Python API, so please revert this change

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@teoparvanov, I think we should rewrite/delete this as the title is Aggregator based but the the tutorial is using python native API.
https://openfl.readthedocs.io/en/latest/developer_guide/running_the_federation.notebook.html

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then my suggestion is to delete the entire page.

**********************************

You will start a Jupyter\* \ lab server and receive a URL you can use to access the tutorials. Jupyter notebooks are provided for PyTorch\* \ and TensorFlow\* \ that simulate a federation on a local machine.
Expand Down
15 changes: 1 addition & 14 deletions docs/developer_guide/utilities/splitters_data.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,20 +13,7 @@ Dataset Splitters
You may apply data splitters differently depending on the |productName| workflow that you follow.


OPTION 1: Use **Native Python API** (Aggregator-Based Workflow) Functions to Split the Data
===========================================================================================

Predefined |productName| data splitters functions are as follows:

- ``openfl.utilities.data_splitters.EqualNumPyDataSplitter`` (default)
- ``openfl.utilities.data_splitters.RandomNumPyDataSplitter``
- ``openfl.interface.aggregation_functions.LogNormalNumPyDataSplitter``, which assumes the ``data`` argument as ``np.ndarray`` of integers (labels)
- ``openfl.interface.aggregation_functions.DirichletNumPyDataSplitter``, which assumes the ``data`` argument as ``np.ndarray`` of integers (labels)

Alternatively, you can create an `implementation <https://github.com/intel/openfl/blob/develop/openfl/utilities/data_splitters/numpy.py>`_ of :class:`openfl.plugins.data_splitters.NumPyDataSplitter` and pass it to the :code:`FederatedDataset` function as either ``train_splitter`` or ``valid_splitter`` keyword argument.


OPTION 2: Use Dataset Splitters in your Shard Descriptor
OPTION 1: Use Dataset Splitters in your Shard Descriptor
========================================================

Apply one of previously mentioned splitting function on your data to perform a simulation.
Expand Down
14 changes: 0 additions & 14 deletions docs/get_started/examples.rst
Original file line number Diff line number Diff line change
Expand Up @@ -29,20 +29,6 @@ See :ref:`running_the_task_runner`

:ref:`running_the_task_runner`

-------------------------
Python Native API
-------------------------
Intended for quick simulation purposes

See :ref:`python_native_pytorch_mnist`

.. toctree::
:hidden:
:maxdepth: 1

examples/python_native_pytorch_mnist


-------------------------
Interactive API
-------------------------
Expand Down
Loading
Loading