Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

updating naming conventions of different API #946

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,13 +37,13 @@ For more installation options check out the [online documentation](https://openf

OpenFL enables data scientists to set up a federated learning experiment following one of the workflows:

- [Director-based Workflow](https://openfl.readthedocs.io/en/latest/about/features_index/interactive.html):
- [Interactive API](https://openfl.readthedocs.io/en/latest/about/features_index/interactive.html):
Setup long-lived components to run many experiments in series. Recommended for FL research when many changes to model, dataloader, or hyperparameters are expected

- [Aggregator-based Workflow](https://openfl.readthedocs.io/en/latest/about/features_index/taskrunner.html):
- [Task Runner API](https://openfl.readthedocs.io/en/latest/about/features_index/taskrunner.html):
Define an experiment and distribute it manually. All participants can verify model code and [FL plan](https://openfl.readthedocs.io/en/latest/about/features_index/taskrunner.html#federated-learning-plan-fl-plan-settings) prior to execution. The federation is terminated when the experiment is finished

- [Workflow Interface](https://openfl.readthedocs.io/en/latest/about/features_index/workflowinterface.html) ([*experimental*](https://openfl.readthedocs.io/en/latest/developer_guide/experimental_features.html)):
- [Workflow API](https://openfl.readthedocs.io/en/latest/about/features_index/workflowinterface.html) ([*experimental*](https://openfl.readthedocs.io/en/latest/developer_guide/experimental_features.html)):
Create complex experiments that extend beyond traditional horizontal federated learning. See the [experimental tutorials](https://github.com/intel/openfl/blob/develop/openfl-tutorials/experimental/) to learn how to coordinate [aggregator validation after collaborator model training](https://github.com/intel/openfl/tree/develop/openfl-tutorials/experimental/Workflow_Interface_102_Aggregator_Validation.ipynb), [perform global differentially private federated learning](https://github.com/psfoley/openfl/tree/experimental-workflow-interface/openfl-tutorials/experimental/Global_DP), measure the amount of private information embedded in a model after collaborator training with [privacy meter](https://github.com/intel/openfl/blob/develop/openfl-tutorials/experimental/Privacy_Meter/readme.md), or [add a watermark to a federated model](https://github.com/intel/openfl/blob/develop/openfl-tutorials/experimental/Workflow_Interface_301_MNIST_Watermarking.ipynb).

The quickest way to test OpenFL is to follow our [tutorials](https://github.com/intel/openfl/tree/develop/openfl-tutorials). </br>
Expand Down
6 changes: 3 additions & 3 deletions docs/about/features.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Running a Federation

|productName| has multiple options for setting up a federation and running experiments, depending on the users needs.

Task Runner
Task Runner API
Define an experiment and distribute it manually. All participants can verify model code and FL plan prior to execution.
The federation is terminated when the experiment is finished. Formerly known as the aggregator-based workflow.
For more info see :doc:`features_index/taskrunner`
Expand All @@ -23,7 +23,7 @@ Task Runner

features_index/taskrunner

Interactive
Interactive API
Setup long-lived components to run many experiments in series. Recommended for FL research when many changes to model, dataloader, or hyperparameters are expected.
Formerly known as the director-based workflow.
For more info see :doc:`features_index/interactive`
Expand All @@ -33,7 +33,7 @@ Interactive

features_index/interactive

Workflow Interface (Experimental)
Workflow API (Experimental)
Formulate the experiment as a series of tasks, or a flow. Every flow begins with the start task and concludes with end.
Heavily influenced by the interface and design of Netflix's Metaflow, the popular framework for data scientists.
For more info see :doc:`features_index/workflowinterface`
Expand Down
8 changes: 4 additions & 4 deletions docs/about/features_index/fed_eval.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Model evaluation is an essential part of the machine learning development cycle.
|productName|'s Support for Federated Evaluation
-------------------------------------------------

|productName|, a flexible framework for Federated Learning, has the capability to perform federated evaluation by modifying the federation plan. In this document, we will show how OpenFL can facilitate this process through its task runner API (aggregator-based workflow), where the model evaluation is distributed across various collaborators before being sent to the aggregator. For the task runner API, this involves minor modifications to the ``plan.yaml`` file, which defines the workflow and tasks for the federation. In particular, the federation plan should be defined to run for one forward pass and perform only aggregated model validation
|productName|, a flexible framework for Federated Learning, has the capability to perform federated evaluation by modifying the federation plan. In this document, we will show how OpenFL can facilitate this process through its Task Runner API, where the model evaluation is distributed across various collaborators before being sent to the aggregator. For the task runner API, this involves minor modifications to the ``plan.yaml`` file, which defines the workflow and tasks for the federation. In particular, the federation plan should be defined to run for one forward pass and perform only aggregated model validation

In general pipeline is as follows:

Expand All @@ -21,10 +21,10 @@ In general pipeline is as follows:
3. **Evaluation**: Each collaborator evaluates the model on its local data.
4. **Aggregation**: The aggregator collects and aggregates these metrics to assess overall model performance.

Example Using the Task Runner API (Aggregator-based Workflow)
--------------------------------------------------------------
Example Using the Task Runner API
----------------------------------

To demonstrate usage of the task runner API (aggregator-based workflow) for federated evaluation, consider the `Hello Federation example <https://github.com/securefederatedai/openfl/blob/develop/tests/github/test_hello_federation.py>`_. This sample script creates a simple federation with two collaborator nodes and one aggregator node, and executes based on a user specified workspace template. We provide a ``torch_cnn_mnist_fed_eval`` template, which is a federated evaluation template adapted from ``torch_cnn_mnist``.
To demonstrate usage of the Task Runner API for federated evaluation, consider the `Hello Federation example <https://github.com/securefederatedai/openfl/blob/develop/tests/github/test_hello_federation.py>`_. This sample script creates a simple federation with two collaborator nodes and one aggregator node, and executes based on a user specified workspace template. We provide a ``torch_cnn_mnist_fed_eval`` template, which is a federated evaluation template adapted from ``torch_cnn_mnist``.

This script can be directly executed as follows:

Expand Down
8 changes: 4 additions & 4 deletions docs/about/features_index/interactive.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,19 +7,19 @@
Interactive API
================

A director-based workflow uses long-lived components in a federation. These components continue to be available to distribute more experiments in the federation.
A Interactive API uses long-lived components in a federation. These components continue to be available to distribute more experiments in the federation.

- The *Director* is the central node of the federation. This component starts an *Aggregator* for each experiment, sends data to connected collaborator nodes, and provides updates on the status.
- The *Envoy* runs on collaborator nodes connected to the *Director*. When the *Director* starts an experiment, the *Envoy* starts the *Collaborator* to train the global model.


The director-based workflow comprises the following roles and their tasks:
The Interactive API comprises the following roles and their tasks:

- `Director Manager: Set Up the Director`_
- `Collaborator Manager: Set Up the Envoy`_
- `Experiment Manager: Describe an Experiment`_

Follow the procedure in the director-based workflow to become familiar with the setup required and APIs provided for each role in the federation: *Experiment manager (Data scientist)*, *Director manager*, and *Collaborator manager*.
Follow the procedure in the Interactive API to become familiar with the setup required and APIs provided for each role in the federation: *Experiment manager (Data scientist)*, *Director manager*, and *Collaborator manager*.

- *Experiment manager* (or Data scientist) is a person or group of people using OpenFL.
- *Director Manager* is ML model creator's representative controlling Director.
Expand All @@ -34,7 +34,7 @@ An overview of this workflow is shown below.

.. figure:: ../../source/openfl/director_workflow.svg

.. centered:: Overview of the Director-Based Workflow
.. centered:: Overview of the Interactive API


.. # Copyright (C) 2020-2023 Intel Corporation
Expand Down
10 changes: 5 additions & 5 deletions docs/about/features_index/workflowinterface.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@
.. _workflow_interface:

******************
Workflow Interface
Workflow API
******************

**Important Note**

The OpenFL workflow interface is experimental, subject to change, and is currently limited to single node execution. To setup and launch a real federation, see :ref:`running_a_federation`
The OpenFL Workflow API is experimental, subject to change, and is currently limited to single node execution. To setup and launch a real federation, see :ref:`running_a_federation`

What is it?
===========
Expand All @@ -30,7 +30,7 @@ There are several modifications we make in our reimagined version of this interf
How to use it?
==============

Let's start with the basics. A flow is intended to define the entirety of federated learning experiment. Every flow begins with the `start` task and concludes with the `end` task. At each step in the flow, attributes can be defined, modified, or deleted. Attributes get passed forward to the next step in the flow, which is defined by the name of the task passed to the `next` function. In the line before each task, there is a **placement decorator**. The placement decorator defines where that task will be run. The OpenFL Workflow Interface adopts the conventions set by Metaflow, that every workflow begins with start and concludes with the end task. In the following example, the aggregator begins with an optionally passed in model and optimizer. The aggregator begins the flow with the start task, where the list of collaborators is extracted from the runtime (:code:`self.collaborators = self.runtime.collaborators`) and is then used as the list of participants to run the task listed in self.next, aggregated_model_validation. The model, optimizer, and anything that is not explicitly excluded from the next function will be passed from the start function on the aggregator to the aggregated_model_validation task on the collaborator. Where the tasks run is determined by the placement decorator that precedes each task definition (:code:`@aggregator` or :code:`@collaborator`). Once each of the collaborators (defined in the runtime) complete the aggregated_model_validation task, they pass their current state onto the train task, from train to local_model_validation, and then finally to join at the aggregator. It is in join that an average is taken of the model weights, and the next round can begin.
Let's start with the basics. A flow is intended to define the entirety of federated learning experiment. Every flow begins with the `start` task and concludes with the `end` task. At each step in the flow, attributes can be defined, modified, or deleted. Attributes get passed forward to the next step in the flow, which is defined by the name of the task passed to the `next` function. In the line before each task, there is a **placement decorator**. The placement decorator defines where that task will be run. The OpenFL Workflow API adopts the conventions set by Metaflow, that every workflow begins with start and concludes with the end task. In the following example, the aggregator begins with an optionally passed in model and optimizer. The aggregator begins the flow with the start task, where the list of collaborators is extracted from the runtime (:code:`self.collaborators = self.runtime.collaborators`) and is then used as the list of participants to run the task listed in self.next, aggregated_model_validation. The model, optimizer, and anything that is not explicitly excluded from the next function will be passed from the start function on the aggregator to the aggregated_model_validation task on the collaborator. Where the tasks run is determined by the placement decorator that precedes each task definition (:code:`@aggregator` or :code:`@collaborator`). Once each of the collaborators (defined in the runtime) complete the aggregated_model_validation task, they pass their current state onto the train task, from train to local_model_validation, and then finally to join at the aggregator. It is in join that an average is taken of the model weights, and the next round can begin.

.. code-block:: python

Expand Down Expand Up @@ -134,10 +134,10 @@ Goals
5. The placement of tasks and how they connect should be straightforward
6. Don't reinvent unless absolutely necessary

Workflow Interface API
Workflow API
======================

The workflow interface formulates the experiment as a series of tasks, or a flow. Every flow begins with the `start` task and concludes with `end`.
The Workflow API formulates the experiment as a series of tasks, or a flow. Every flow begins with the `start` task and concludes with `end`.

Runtimes
========
Expand Down
4 changes: 2 additions & 2 deletions docs/developer_guide/advanced_topics.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Advanced Topics

- :doc:`advanced_topics/bash_autocomplete_activation`

**Aggregator-Based Workflow**
**Task Runner API**

Learn to manage multiple Federation Learning plans (FL plan) in the same workspace:

Expand All @@ -30,7 +30,7 @@ Advanced Topics

- :doc:`advanced_topics/straggler_handling_algorithms`

**Director-Based Workflow**
**Interactive API**

Customize the logging function for each task:

Expand Down
2 changes: 1 addition & 1 deletion docs/developer_guide/advanced_topics/multiple_plans.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
Manage Multiple Plans
***********************

With aggregator-based workflow, you can use multiple Federated Learning plans (FL plan) for the same workspace. All FL plans are located in the **WORKSPACE.FOLDER/plan/plans** directory.
With the Task Runner API, you can use multiple Federated Learning plans (FL plan) for the same workspace. All FL plans are located in the **WORKSPACE.FOLDER/plan/plans** directory.

The following are the :code:`fx` commands to manage your FL plans:

Expand Down
2 changes: 1 addition & 1 deletion docs/developer_guide/advanced_topics/overriding_agg_fn.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
Override Aggregation Function
*****************************

With the aggregator-based workflow, you can use custom aggregation functions for each task via Python\*\ API or command line interface.
With the Task Runner API, you can use custom aggregation functions for each task via Python\*\ API or command line interface.


Python API
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
Updating plan settings
***********************

With the director-based workflow, you can use custom plan settings before starting the experiment. Changing plan settings in command line interface is straightforward by modifying plan.yaml.
With the Interactive API, you can use custom plan settings before starting the experiment. Changing plan settings in command line interface is straightforward by modifying plan.yaml.
When using Python API or Director Envoy based interactive API, **override_config** can be used to update plan settings.


Expand Down
3 changes: 2 additions & 1 deletion docs/developer_guide/experimental_features.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,14 @@ Experimental features are *not* ready for production. These features are under a
1. *Backward compatibility is not guaranteed* - Our goal is to maintain backward compatibility whenever possible, but user feedback (and our own internal research)
may result in necessary changes to the APIs.

**Workflow Interface**
**Workflow API**

Learn how to:
- Chain a series of tasks that run on aggregator or collaborator.
- Filter out information that should stay local
- Use Metaflow tools to analyze and debug experiments


- :doc:`../about/features_index/workflowinterface`

.. toctree::
Expand Down
6 changes: 3 additions & 3 deletions docs/developer_guide/running_the_federation.notebook.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@
.. _running_notebook:

**********************************
Aggregator-Based Workflow Tutorial
Task Runner API Tutorial
**********************************

You will start a Jupyter\* \ lab server and receive a URL you can use to access the tutorials. Jupyter notebooks are provided for PyTorch\* \ and TensorFlow\* \ that simulate a federation on a local machine.

.. note::

Follow the procedure to become familiar with the APIs used in aggregator-based workflow and conventions such as *FL Plans*, *Aggregators*, and *Collaborators*.
Follow the procedure to become familiar with the APIs used in the Task Runner API and conventions such as *FL Plans*, *Aggregators*, and *Collaborators*.


Start the Tutorials
Expand Down Expand Up @@ -41,7 +41,7 @@ Start the Tutorials
- :code:`Federated PyTorch TinyImageNet`: workspace with a MobileNet-V2 `PyTorch <https://pytorch.org/>`_ model that will download the `Tiny-ImageNet <https://www.kaggle.com/c/tiny-imagenet/>`_ dataset and train in a federation.


Familiarize with the API Concepts in an Aggregator-Based Worklow
Familiarize with the API Concepts in the Task Runner API
================================================================

Step 1: Enable the |productName| Python API
Expand Down
8 changes: 4 additions & 4 deletions docs/developer_guide/running_the_federation_with_gandlf.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,20 +10,20 @@ Run the Federation with a model defined using GaNDLF
This guide will show you how to take an existing model using the `Generally Nuanced Deep Learning Framework (GaNDLF) <https://github.com/mlcommons/GaNDLF>`_ experiment to a federated environment.


`Aggregator-Based Workflow`_
`Task Runner API`_
Define an experiment and distribute it manually. All participants can verify model code and `FL plan <https://openfl.readthedocs.io/en/latest/running_the_federation.html#federated-learning-plan-fl-plan-settings>`_ prior to executing the code/model. The federation is terminated when the experiment is finished, and appropriate statistics are generated.


.. _running_the_federation_aggregator_based_gandlf:

Aggregator-Based Workflow
Task Runner API
=========================

An overview of this workflow is shown below.

.. figure:: ../images/openfl_flow.png

.. centered:: Overview of the Aggregator-Based Workflow
.. centered:: Overview of the Task Runner API


This workflow uses short-lived components in a federation, which is terminated when the experiment is finished. The components are as follows:
Expand Down Expand Up @@ -91,7 +91,7 @@ You can use the `"Hello Federation" bash script <https://github.com/intel/openfl
.. literalinclude:: ../../tests/github/test_hello_federation.py
:language: bash

However, continue with the following procedure for details in creating a federation with an aggregator-based workflow.
However, continue with the following procedure for details in creating a federation with the Task Runner API.

`STEP 1: Install GaNDLF prerequisites and Create a Workspace`_

Expand Down
Loading
Loading