From f239af842f359fe623f331e95d676b1293bc12dc Mon Sep 17 00:00:00 2001 From: dstripelis Date: Mon, 18 Nov 2024 21:07:53 +0200 Subject: [PATCH 01/26] Updating how to upgrade to flower next documentation by also including examples for Flower versions 1.11 and above. --- doc/source/how-to-upgrade-to-flower-next.rst | 81 +++++++++++++++----- 1 file changed, 60 insertions(+), 21 deletions(-) diff --git a/doc/source/how-to-upgrade-to-flower-next.rst b/doc/source/how-to-upgrade-to-flower-next.rst index 9a476f9865e1..9b01467c7453 100644 --- a/doc/source/how-to-upgrade-to-flower-next.rst +++ b/doc/source/how-to-upgrade-to-flower-next.rst @@ -124,9 +124,12 @@ In Flower Next, the *infrastructure* and *application layers* have been decouple Instead of starting a client in code via ``start_client()``, you create a |clientapp_link|_ and start it via the command line. Instead of starting a server in code via ``start_server()``, you create a |serverapp_link|_ and start it via the command -line. The long-running components of server and client are called SuperLink and -SuperNode. The following non-breaking changes that require manual updates and allow you -to run your project both in the traditional way and in the Flower Next way: +line. The long-running components of server and client are called `SuperLink` and +`SuperNode`, for more details please see the `Flower Architecture`_. The following +non-breaking changes that require manual updates and allow you to run your project both +in the traditional way and in the Flower Next way: + +.. _flower architecture: explanation-flower-architecture.rst |clientapp_link|_ ~~~~~~~~~~~~~~~~~ @@ -135,9 +138,19 @@ to run your project both in the traditional way and in the Flower Next way: |startclient_link|_. Here's an example: .. code-block:: python - :emphasize-lines: 5,11 + :emphasize-lines: 2,6 + + # Flower v1.11+ + def client_fn(context: flwr.common.Context): + return flwr.client.FlowerClient().to_client() + + + app = flwr.client.ClientApp( + client_fn=client_fn, + ) + - # Flower 1.8 + # Flower v1.8 - v1.10 def client_fn(cid: str): return flwr.client.FlowerClient().to_client() @@ -146,7 +159,7 @@ to run your project both in the traditional way and in the Flower Next way: client_fn=client_fn, ) - # Flower 1.7 + # Flower v1.7 if __name__ == "__main__": flwr.client.start_client( server_address="127.0.0.1:8080", @@ -160,15 +173,26 @@ to run your project both in the traditional way and in the Flower Next way: |startserver_link|_. Here's an example: .. code-block:: python - :emphasize-lines: 2,9 + :emphasize-lines: 2,5,11 + + # Flower v1.11+ + def server_fn(context: flwr.common.Context): + strategy = flwr.server.strategy.FedAvg() + config = flwr.server.ServerConfig() + return flwr.server.ServerAppComponents(strategy=strategy, config=config) + + + app = flwr.server.ServerApp( + server_fn=server_fn, + ) - # Flower 1.8 + # Flower v1.8 - v1.11 app = flwr.server.ServerApp( config=config, strategy=strategy, ) - # Flower 1.7 + # Flower v1.7 if __name__ == "__main__": flwr.server.start_server( server_address="0.0.0.0:8080", @@ -232,7 +256,7 @@ Simulation in CLI respectively. There is no need to use |startsim_link|_ anymore. Here's an example: .. code-block:: python - :emphasize-lines: 9,13,20 + :emphasize-lines: 7,11,14 # Regular Flower client implementation class FlowerClient(NumPyClient): @@ -240,7 +264,22 @@ Simulation in CLI pass - # Flower 1.8 + # Flower v1.11+ + def client_fn(context: flwr.common.Context): + return flwr.client.FlowerClient().to_client() + + + app = flwr.client.ClientApp( + client_fn=client_fn, + ) + + server_app = flwr.server.ServerApp( + config=config, + strategy=strategy, + ) + + + # Flower v1.8 - v1.10 def client_fn(cid: str): return FlowerClient().to_client() @@ -254,7 +293,7 @@ Simulation in CLI strategy=strategy, ) - # Flower 1.7 + # Flower v1.7 if __name__ == "__main__": hist = flwr.simulation.start_simulation( num_clients=100, @@ -267,7 +306,7 @@ Simulation in CLI .. code-block:: bash - # Flower 1.8 + # Flower v1.8 $ flower-simulation \ --server-app=sim:server_app \ --client-app=sim:client_app \ @@ -275,7 +314,7 @@ Simulation in CLI .. code-block:: bash - # Flower 1.7 + # Flower v1.7 $ python sim.py - Set default resources for each |clientapp_link|_ using the ``--backend-config`` @@ -283,9 +322,9 @@ Simulation in CLI |startsim_link|_. Here's an example: .. code-block:: bash - :emphasize-lines: 6 + :emphasize-lines: - # Flower 1.8 + # Flower v1.8 $ flower-simulation \ --client-app=sim:client_app \ --server-app=sim:server_app \ @@ -293,9 +332,9 @@ Simulation in CLI --backend-config='{"client_resources": {"num_cpus": 2, "num_gpus": 0.25}}' .. code-block:: python - :emphasize-lines: 5 + :emphasize-lines: - # Flower 1.7 (in `sim.py`) + # Flower v1.7 (in `sim.py`) if __name__ == "__main__": hist = flwr.simulation.start_simulation( num_clients=100, client_resources={"num_cpus": 2, "num_gpus": 0.25}, ... @@ -307,7 +346,7 @@ Simulation in a Notebook - Run |runsim_link|_ in your notebook instead of |startsim_link|_. Here's an example: .. code-block:: python - :emphasize-lines: 19,27 + :emphasize-lines: 21 NUM_CLIENTS = 10 # Replace by any integer greater than zero @@ -328,7 +367,7 @@ Simulation in a Notebook backend_config = {"client_resources": {"num_cpus": 2, "num_gpus": 0.25}} - # Flower 1.8 + # Flower v1.8 flwr.simulation.run_simulation( server_app=server_app, client_app=client_app, @@ -336,7 +375,7 @@ Simulation in a Notebook backend_config=backend_config, ) - # Flower 1.7 + # Flower v1.7 flwr.simulation.start_simulation( client_fn=client_fn, num_clients=NUM_CLIENTS, From 31999bc39b2d12692663fd6cf7c789d264ef1ba7 Mon Sep 17 00:00:00 2001 From: dstripelis Date: Mon, 18 Nov 2024 21:14:13 +0200 Subject: [PATCH 02/26] Moving below highlighting in simulation CLI. --- doc/source/how-to-upgrade-to-flower-next.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/source/how-to-upgrade-to-flower-next.rst b/doc/source/how-to-upgrade-to-flower-next.rst index 9b01467c7453..c768b39aacae 100644 --- a/doc/source/how-to-upgrade-to-flower-next.rst +++ b/doc/source/how-to-upgrade-to-flower-next.rst @@ -256,7 +256,7 @@ Simulation in CLI respectively. There is no need to use |startsim_link|_ anymore. Here's an example: .. code-block:: python - :emphasize-lines: 7,11,14 + :emphasize-lines: 8,13,16 # Regular Flower client implementation class FlowerClient(NumPyClient): From 7aa73cb0dc95c304b1280bedfe2f469e5d47d21a Mon Sep 17 00:00:00 2001 From: dstripelis Date: Tue, 19 Nov 2024 15:11:53 +0200 Subject: [PATCH 03/26] Addressing comments and expanding and clarifying the guidelines for the different flower simulation versions. --- doc/source/how-to-upgrade-to-flower-next.rst | 136 ++++++++++++++----- 1 file changed, 105 insertions(+), 31 deletions(-) diff --git a/doc/source/how-to-upgrade-to-flower-next.rst b/doc/source/how-to-upgrade-to-flower-next.rst index c768b39aacae..1b7ed3c1418d 100644 --- a/doc/source/how-to-upgrade-to-flower-next.rst +++ b/doc/source/how-to-upgrade-to-flower-next.rst @@ -37,10 +37,18 @@ Let's dive in! .. |flowernext_serverapp_link| replace:: ``flower-server-app`` +.. |flower_architecture_link| replace:: Flower Architecture + +.. |flower_how_to_run_simulations_link| replace:: How-to Run Simulations + .. |flower_simulation_link| replace:: ``flower-simulation`` .. _clientapp_link: ref-api/flwr.client.ClientApp.html +.. _flower_architecture_link: explanation-flower-architecture.html + +.. _flower_how_to_run_simulations_link: how-to-run-simulations.html + .. _flower_simulation_link: ref-api-cli.html#flower-simulation .. _flowernext_clientapp_link: ref-api-cli.html#flower-client-app @@ -125,12 +133,10 @@ Instead of starting a client in code via ``start_client()``, you create a |clientapp_link|_ and start it via the command line. Instead of starting a server in code via ``start_server()``, you create a |serverapp_link|_ and start it via the command line. The long-running components of server and client are called `SuperLink` and -`SuperNode`, for more details please see the `Flower Architecture`_. The following +`SuperNode`, for more details please see the |flower_architecture_link|_ . The following non-breaking changes that require manual updates and allow you to run your project both in the traditional way and in the Flower Next way: -.. _flower architecture: explanation-flower-architecture.rst - |clientapp_link|_ ~~~~~~~~~~~~~~~~~ @@ -173,7 +179,7 @@ in the traditional way and in the Flower Next way: |startserver_link|_. Here's an example: .. code-block:: python - :emphasize-lines: 2,5,11 + :emphasize-lines: 2,8 # Flower v1.11+ def server_fn(context: flwr.common.Context): @@ -252,11 +258,11 @@ Deployment Simulation in CLI ~~~~~~~~~~~~~~~~~ -- Wrap your existing client and strategy with |clientapp_link|_ and |serverapp_link|_, - respectively. There is no need to use |startsim_link|_ anymore. Here's an example: +Wrap your existing client and strategy with |clientapp_link|_ and |serverapp_link|_, +respectively. There is no need to use |startsim_link|_ anymore. Here's an example: .. code-block:: python - :emphasize-lines: 8,13,16 + :emphasize-lines: 9,13,18,25 # Regular Flower client implementation class FlowerClient(NumPyClient): @@ -265,6 +271,7 @@ Simulation in CLI # Flower v1.11+ + # [file: client_app.py] def client_fn(context: flwr.common.Context): return flwr.client.FlowerClient().to_client() @@ -273,9 +280,16 @@ Simulation in CLI client_fn=client_fn, ) + + # [file: server_app.py] + def server_fn(context: flwr.common.Context): + strategy = flwr.server.strategy.FedAvg(...) + config = flwr.server.ServerConfig(...) + return flwr.server.ServerAppComponents(strategy=strategy, config=config) + + server_app = flwr.server.ServerApp( - config=config, - strategy=strategy, + server_fn=server_fn, ) @@ -296,59 +310,117 @@ Simulation in CLI # Flower v1.7 if __name__ == "__main__": hist = flwr.simulation.start_simulation( - num_clients=100, + num_clients=10, # ... ) -- Run |flower_simulation_link|_ in CLI and point to the ``server_app`` / ``client_app`` - object in the code instead of executing the Python script. Here's an example (assuming - the ``server_app`` and ``client_app`` objects are in a ``sim.py`` module): +Depending on your Flower version, you can run your simulation as follows: + +- for Flower versions 1.11 and onwards, run ``flwr run`` in CLI. +- for Flower versions between 1.8 to 1.10, run |flower_simulation_link|_ in CLI and + point to the ``server_app`` / ``client_app`` object in the code instead of executing + the Python script. In the code snippet below, there is an example (assuming the + ``server_app`` and ``client_app`` objects are in a ``sim.py`` module). +- for Flower versions before 1.8, run the Python script directly. .. code-block:: bash + :emphasize-lines: 2 - # Flower v1.8 + # Flower v1.11+ + $ flwr run + + + # Flower v1.8 - v1.10 $ flower-simulation \ --server-app=sim:server_app \ --client-app=sim:client_app \ - --num-supernodes=100 + --num-supernodes=10 -.. code-block:: bash # Flower v1.7 $ python sim.py -- Set default resources for each |clientapp_link|_ using the ``--backend-config`` - command line argument instead of setting the ``client_resources`` argument in - |startsim_link|_. Here's an example: +Depending on your Flower version, you can also define the default resources as follows: + +- for Flower versions 1.11 and onwards, you can edit your pyproject.toml file and then + run ``flwr run`` in CLI as shown in the example below. +- for Flower versions between 1.8 to 1.10, you can adjust the resources for each + |clientapp_link|_ using the ``--backend-config`` command line argument instead of + setting the ``client_resources`` argument in |startsim_link|_. +- for Flower versions before 1.8, you need to run |startsim_link|_ and pass a dictionary + of the required resources to the ``client_resources`` argument. .. code-block:: bash - :emphasize-lines: + :emphasize-lines: 2,8 + + # Flower v.1.11+ + # pyproject.toml + [tool.flwr.federations.local-sim-gpu] + options.num-supernodes = 10 + options.backend.client-resources.num-cpus = 2 + options.backend.client-resources.num-gpus = 0.25 - # Flower v1.8 + $ flwr run + + # Flower v1.8 - v1.10 $ flower-simulation \ --client-app=sim:client_app \ --server-app=sim:server_app \ - --num-supernodes=100 \ + --num-supernodes=10 \ --backend-config='{"client_resources": {"num_cpus": 2, "num_gpus": 0.25}}' -.. code-block:: python - :emphasize-lines: - # Flower v1.7 (in `sim.py`) if __name__ == "__main__": hist = flwr.simulation.start_simulation( - num_clients=100, client_resources={"num_cpus": 2, "num_gpus": 0.25}, ... + num_clients=10, client_resources={"num_cpus": 2, "num_gpus": 0.25}, ... ) Simulation in a Notebook ~~~~~~~~~~~~~~~~~~~~~~~~ -- Run |runsim_link|_ in your notebook instead of |startsim_link|_. Here's an example: +To run your simulation from within a notebook, please consider the following examples +depending on your Flower version: + +- for Flower versions 1.11 and onwards, you first need to edit your pyproject.toml file + as shown in the example below and run |runsim_link|_ in your notebook instead of + |startsim_link|_. +- for Flower versions between 1.8 to 1.10, you need to run |runsim_link|_ in your + notebook instead of |startsim_link|_ and configure the resources. +- for Flower versions before 1.8, you need to run |startsim_link|_ and pass a dictionary + of the required resources to the ``client_resources`` argument. .. code-block:: python - :emphasize-lines: 21 + :emphasize-lines: 2,9,13,17,21 + # Flower v1.11+ + # pyproject.toml + [tool.flwr.federations.local - sim - gpu] + options.num - supernodes = 10 + options.backend.client - resources.num - cpus = 2 + options.backend.client - resources.num - gpus = 0.25 + + + def client_fn(context: flwr.common.Context): + return flwr.client.FlowerClient().to_client() + + + client_app = flwr.server.ClientApp( + client_fn=client_fn, + ) + + server_app = flwr.server.ServerApp( + server_fn=server_fn, + ) + + flwr.simulation.run_simulation( + server_app=server_app, + client_app=client_app, + ) + + + # Flower v1.8 - v1.10 NUM_CLIENTS = 10 # Replace by any integer greater than zero + backend_config = {"client_resources": {"num_cpus": 2, "num_gpus": 0.25}} def client_fn(cid: str): @@ -365,9 +437,6 @@ Simulation in a Notebook strategy=strategy, ) - backend_config = {"client_resources": {"num_cpus": 2, "num_gpus": 0.25}} - - # Flower v1.8 flwr.simulation.run_simulation( server_app=server_app, client_app=client_app, @@ -376,6 +445,8 @@ Simulation in a Notebook ) # Flower v1.7 + NUM_CLIENTS = 10 # Replace by any integer greater than zero + backend_config = {"client_resources": {"num_cpus": 2, "num_gpus": 0.25}} flwr.simulation.start_simulation( client_fn=client_fn, num_clients=NUM_CLIENTS, @@ -384,6 +455,9 @@ Simulation in a Notebook client_resources=backend_config["client_resources"], ) +For more advanced information regarding Flower simulation please read the +|flower_how_to_run_simulations_link|_ guide. + Further help ------------ From c134b1802ec4bfeb5aa922953b2ef27c88123a88 Mon Sep 17 00:00:00 2001 From: dstripelis Date: Tue, 19 Nov 2024 18:15:03 +0200 Subject: [PATCH 04/26] Updating deployment section to version 1.13. --- doc/source/how-to-upgrade-to-flower-next.rst | 48 ++++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/doc/source/how-to-upgrade-to-flower-next.rst b/doc/source/how-to-upgrade-to-flower-next.rst index 1b7ed3c1418d..98310a0ae63d 100644 --- a/doc/source/how-to-upgrade-to-flower-next.rst +++ b/doc/source/how-to-upgrade-to-flower-next.rst @@ -219,6 +219,30 @@ Deployment # Start a Superlink $ flower-superlink --insecure + # Flower v1.13 + # In a new terminal window, start a long-running SuperNode + $ flower-supernode \ + --insecure \ + --superlink 127.0.0.1:9092 \ + --node-config "partition-id=0 num-partitions=2" \ + --supernode-address 127.0.0.1:9094 \ + --isolation subprocess + + # In another terminal window, start another long-running SuperNode (at least 2 SuperNodes are required) + $ flower-supernode \ + --insecure \ + --superlink 127.0.0.1:9092 \ + --node-config "partition-id=1 num-partitions=2" \ + --supernode-address 127.0.0.1:9095 \ + --isolation subprocess + + # In a new terminal, start the SuperExec process with the following command: + $ flower-superexec \ + --insecure \ + --executor flwr.superexec.deployment:executor \ + --executor-config 'superlink="127.0.0.1:9091"' + + # Flower up to v1.12 # In a new terminal window, start a long-running SuperNode $ flower-client-app client:app --insecure @@ -240,6 +264,30 @@ Deployment --ssl-certfile \ --ssl-keyfile + # Flower v1.13 + # In a new terminal window, start a long-running SuperNode + $ flower-supernode \ + --superlink 127.0.0.1:9092 \ + --node-config "partition-id=0 num-partitions=2" \ + --supernode-address 127.0.0.1:9094 \ + --isolation subprocess \ + --root-certificates + + # In another terminal window, start another long-running SuperNode (at least 2 SuperNodes are required) + $ flower-supernode \ + --superlink 127.0.0.1:9092 \ + --node-config "partition-id=1 num-partitions=2" \ + --supernode-address 127.0.0.1:9095 \ + --isolation subprocess \ + --root-certificates + + # In a new terminal, start the SuperExec process with the following command: + $ flower-superexec \ + --executor flwr.superexec.deployment:executor \ + --executor-config 'superlink="127.0.0.1:9091"' \ + --root-certificates + + # Flower up to v1.12 # In a new terminal window, start a long-running secure SuperNode $ flower-client-app client:app \ --root-certificates \ From a7b30c58fd94dfe10531a235f49fc4f7565766bd Mon Sep 17 00:00:00 2001 From: dstripelis Date: Tue, 19 Nov 2024 18:19:47 +0200 Subject: [PATCH 05/26] Adding emphasizing lines in the code segment of the Deployment section. --- doc/source/how-to-upgrade-to-flower-next.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/source/how-to-upgrade-to-flower-next.rst b/doc/source/how-to-upgrade-to-flower-next.rst index 98310a0ae63d..3cb99d481aac 100644 --- a/doc/source/how-to-upgrade-to-flower-next.rst +++ b/doc/source/how-to-upgrade-to-flower-next.rst @@ -215,6 +215,7 @@ Deployment - Here's an example to start the server without HTTPS (only for prototyping): .. code-block:: bash + :emphasize-lines: 2,6,14,22 # Start a Superlink $ flower-superlink --insecure @@ -257,6 +258,7 @@ Deployment certificate, server certificate, and server private key). .. code-block:: bash + :emphasize-lines: 2,9,17,25 # Start a secure Superlink $ flower-superlink \ From 50ce507c8f2d7dc48aadcc9fb4f48be37f131c8d Mon Sep 17 00:00:00 2001 From: dstripelis Date: Tue, 19 Nov 2024 18:48:03 +0200 Subject: [PATCH 06/26] Simplifying deployment by keeping only flower-superlink and flower-supernode and adding respective api references. --- doc/source/how-to-upgrade-to-flower-next.rst | 60 ++++---------------- 1 file changed, 12 insertions(+), 48 deletions(-) diff --git a/doc/source/how-to-upgrade-to-flower-next.rst b/doc/source/how-to-upgrade-to-flower-next.rst index 3cb99d481aac..bd15488289e3 100644 --- a/doc/source/how-to-upgrade-to-flower-next.rst +++ b/doc/source/how-to-upgrade-to-flower-next.rst @@ -37,6 +37,10 @@ Let's dive in! .. |flowernext_serverapp_link| replace:: ``flower-server-app`` +.. |flowernext_superlink_link| replace:: ``flower-superlink`` + +.. |flowernext_supernode_link| replace:: ``flower-supernode`` + .. |flower_architecture_link| replace:: Flower Architecture .. |flower_how_to_run_simulations_link| replace:: How-to Run Simulations @@ -57,6 +61,8 @@ Let's dive in! .. _flowernext_superlink_link: ref-api-cli.html#flower-superlink +.. _flowernext_supernode_link: ref-api-cli.html#flower-supernode + .. _runsim_link: ref-api/flwr.simulation.run_simulation.html .. _serverapp_link: ref-api/flwr.server.ServerApp.html @@ -210,55 +216,36 @@ Deployment ~~~~~~~~~~ - Run the ``SuperLink`` using |flowernext_superlink_link|_ before running, in sequence, - |flowernext_clientapp_link|_ (2x) and |flowernext_serverapp_link|_. There is no need - to execute `client.py` and `server.py` as Python scripts. + |flowernext_supernode_link|_ (2x). - Here's an example to start the server without HTTPS (only for prototyping): .. code-block:: bash - :emphasize-lines: 2,6,14,22 + :emphasize-lines: 2,5,12 # Start a Superlink $ flower-superlink --insecure - # Flower v1.13 # In a new terminal window, start a long-running SuperNode $ flower-supernode \ --insecure \ --superlink 127.0.0.1:9092 \ --node-config "partition-id=0 num-partitions=2" \ - --supernode-address 127.0.0.1:9094 \ - --isolation subprocess + --supernode-address 127.0.0.1:9094 # In another terminal window, start another long-running SuperNode (at least 2 SuperNodes are required) $ flower-supernode \ --insecure \ --superlink 127.0.0.1:9092 \ --node-config "partition-id=1 num-partitions=2" \ - --supernode-address 127.0.0.1:9095 \ - --isolation subprocess + --supernode-address 127.0.0.1:9095 - # In a new terminal, start the SuperExec process with the following command: - $ flower-superexec \ - --insecure \ - --executor flwr.superexec.deployment:executor \ - --executor-config 'superlink="127.0.0.1:9091"' - - # Flower up to v1.12 - # In a new terminal window, start a long-running SuperNode - $ flower-client-app client:app --insecure - - # In another terminal window, start another long-running SuperNode (at least 2 SuperNodes are required) - $ flower-client-app client:app --insecure - - # In yet another terminal window, run the ServerApp (this starts the actual training run) - $ flower-server-app server:app --insecure - Here's another example to start with HTTPS. Use the ``--ssl-ca-certfile``, ``--ssl-certfile``, and ``--ssl-keyfile`` command line options to pass paths to (CA certificate, server certificate, and server private key). .. code-block:: bash - :emphasize-lines: 2,9,17,25 + :emphasize-lines: 2,8,15 # Start a secure Superlink $ flower-superlink \ @@ -266,13 +253,11 @@ Deployment --ssl-certfile \ --ssl-keyfile - # Flower v1.13 # In a new terminal window, start a long-running SuperNode $ flower-supernode \ --superlink 127.0.0.1:9092 \ --node-config "partition-id=0 num-partitions=2" \ --supernode-address 127.0.0.1:9094 \ - --isolation subprocess \ --root-certificates # In another terminal window, start another long-running SuperNode (at least 2 SuperNodes are required) @@ -280,30 +265,9 @@ Deployment --superlink 127.0.0.1:9092 \ --node-config "partition-id=1 num-partitions=2" \ --supernode-address 127.0.0.1:9095 \ - --isolation subprocess \ --root-certificates - # In a new terminal, start the SuperExec process with the following command: - $ flower-superexec \ - --executor flwr.superexec.deployment:executor \ - --executor-config 'superlink="127.0.0.1:9091"' \ - --root-certificates - - # Flower up to v1.12 - # In a new terminal window, start a long-running secure SuperNode - $ flower-client-app client:app \ - --root-certificates \ - --superlink 127.0.0.1:9092 - - # In another terminal window, start another long-running secure SuperNode (at least 2 SuperNodes are required) - $ flower-client-app client:app \ - --root-certificates \ - --superlink 127.0.0.1:9092 - - # In yet another terminal window, run the ServerApp (this starts the actual training run) - $ flower-server-app server:app \ - --root-certificates \ - --superlink 127.0.0.1:9091 + Simulation in CLI ~~~~~~~~~~~~~~~~~ From dc01aad8493657d8288d79fdb9a122d057eff906 Mon Sep 17 00:00:00 2001 From: dstripelis Date: Tue, 19 Nov 2024 21:32:15 +0200 Subject: [PATCH 07/26] Removing duplicate substitution definition. --- doc/source/how-to-upgrade-to-flower-next.rst | 2 -- 1 file changed, 2 deletions(-) diff --git a/doc/source/how-to-upgrade-to-flower-next.rst b/doc/source/how-to-upgrade-to-flower-next.rst index bd15488289e3..925f4b4742c1 100644 --- a/doc/source/how-to-upgrade-to-flower-next.rst +++ b/doc/source/how-to-upgrade-to-flower-next.rst @@ -31,8 +31,6 @@ Let's dive in! .. |runsim_link| replace:: ``run_simulation()`` -.. |flowernext_superlink_link| replace:: ``flower-superlink`` - .. |flowernext_clientapp_link| replace:: ``flower-client-app`` .. |flowernext_serverapp_link| replace:: ``flower-server-app`` From 390d56a6688a88bb8e5fdf34fcfa86a90200a3c2 Mon Sep 17 00:00:00 2001 From: dstripelis Date: Tue, 19 Nov 2024 21:47:10 +0200 Subject: [PATCH 08/26] Minor reformat. --- doc/source/how-to-upgrade-to-flower-next.rst | 3 --- 1 file changed, 3 deletions(-) diff --git a/doc/source/how-to-upgrade-to-flower-next.rst b/doc/source/how-to-upgrade-to-flower-next.rst index 925f4b4742c1..3c5efbb98a49 100644 --- a/doc/source/how-to-upgrade-to-flower-next.rst +++ b/doc/source/how-to-upgrade-to-flower-next.rst @@ -237,7 +237,6 @@ Deployment --node-config "partition-id=1 num-partitions=2" \ --supernode-address 127.0.0.1:9095 - - Here's another example to start with HTTPS. Use the ``--ssl-ca-certfile``, ``--ssl-certfile``, and ``--ssl-keyfile`` command line options to pass paths to (CA certificate, server certificate, and server private key). @@ -265,8 +264,6 @@ Deployment --supernode-address 127.0.0.1:9095 \ --root-certificates - - Simulation in CLI ~~~~~~~~~~~~~~~~~ From 25e72dd66891911907a85db0a605c4d23dd86f84 Mon Sep 17 00:00:00 2001 From: dstripelis Date: Wed, 20 Nov 2024 10:54:55 +0200 Subject: [PATCH 09/26] Addressing Javier's comments. --- doc/source/how-to-upgrade-to-flower-next.rst | 60 ++++++-------------- 1 file changed, 17 insertions(+), 43 deletions(-) diff --git a/doc/source/how-to-upgrade-to-flower-next.rst b/doc/source/how-to-upgrade-to-flower-next.rst index 3c5efbb98a49..9ac94e57988c 100644 --- a/doc/source/how-to-upgrade-to-flower-next.rst +++ b/doc/source/how-to-upgrade-to-flower-next.rst @@ -74,9 +74,6 @@ Let's dive in! Install update -------------- -Using pip -~~~~~~~~~ - Here's how to update an existing installation of Flower to Flower Next with ``pip``: .. code-block:: bash @@ -89,8 +86,6 @@ or if you need Flower Next with simulation: $ python -m pip install -U "flwr[simulation]" -Ensure you set the following version constraint in your ``requirements.txt`` - .. code-block:: # Without simulation support @@ -109,26 +104,6 @@ or ``pyproject.toml``: # With simulation support dependencies = ["flwr[simulation]>=1.8,2.0"] -Using Poetry -~~~~~~~~~~~~ - -Update the ``flwr`` dependency in ``pyproject.toml`` and then reinstall (don't forget to -delete ``poetry.lock`` via ``rm poetry.lock`` before running ``poetry install``). - -Ensure you set the following version constraint in your ``pyproject.toml``: - -.. code-block:: toml - :substitutions: - - [tool.poetry.dependencies] - python = "^|python_version|" - - # Without simulation support - flwr = ">=1.8,<2.0" - - # With simulation support - flwr = { version = ">=1.8,<2.0", extras = ["simulation"] } - Required changes ---------------- @@ -227,14 +202,14 @@ Deployment $ flower-supernode \ --insecure \ --superlink 127.0.0.1:9092 \ - --node-config "partition-id=0 num-partitions=2" \ + --node-config "..." \ --supernode-address 127.0.0.1:9094 # In another terminal window, start another long-running SuperNode (at least 2 SuperNodes are required) $ flower-supernode \ --insecure \ --superlink 127.0.0.1:9092 \ - --node-config "partition-id=1 num-partitions=2" \ + --node-config "..." \ --supernode-address 127.0.0.1:9095 - Here's another example to start with HTTPS. Use the ``--ssl-ca-certfile``, @@ -253,14 +228,14 @@ Deployment # In a new terminal window, start a long-running SuperNode $ flower-supernode \ --superlink 127.0.0.1:9092 \ - --node-config "partition-id=0 num-partitions=2" \ + --node-config "..." \ --supernode-address 127.0.0.1:9094 \ --root-certificates # In another terminal window, start another long-running SuperNode (at least 2 SuperNodes are required) $ flower-supernode \ --superlink 127.0.0.1:9092 \ - --node-config "partition-id=1 num-partitions=2" \ + --node-config "..." \ --supernode-address 127.0.0.1:9095 \ --root-certificates @@ -270,6 +245,11 @@ Simulation in CLI Wrap your existing client and strategy with |clientapp_link|_ and |serverapp_link|_, respectively. There is no need to use |startsim_link|_ anymore. Here's an example: +.. tip:: + + For more advanced information regarding Flower simulation please read the + |flower_how_to_run_simulations_link|_ guide. + .. code-block:: python :emphasize-lines: 9,13,18,25 @@ -390,25 +370,22 @@ Simulation in a Notebook To run your simulation from within a notebook, please consider the following examples depending on your Flower version: -- for Flower versions 1.11 and onwards, you first need to edit your pyproject.toml file - as shown in the example below and run |runsim_link|_ in your notebook instead of - |startsim_link|_. +- for Flower versions 1.11 and onwards, you need to run |runsim_link|_ in your notebook + instead of |startsim_link|_. - for Flower versions between 1.8 to 1.10, you need to run |runsim_link|_ in your notebook instead of |startsim_link|_ and configure the resources. - for Flower versions before 1.8, you need to run |startsim_link|_ and pass a dictionary of the required resources to the ``client_resources`` argument. -.. code-block:: python - :emphasize-lines: 2,9,13,17,21 +.. tip:: - # Flower v1.11+ - # pyproject.toml - [tool.flwr.federations.local - sim - gpu] - options.num - supernodes = 10 - options.backend.client - resources.num - cpus = 2 - options.backend.client - resources.num - gpus = 0.25 + For more advanced information regarding Flower simulation please read the + |flower_how_to_run_simulations_link|_ guide. +.. code-block:: python + :emphasize-lines: 2,6,10,14 + # Flower v1.11+ def client_fn(context: flwr.common.Context): return flwr.client.FlowerClient().to_client() @@ -464,9 +441,6 @@ depending on your Flower version: client_resources=backend_config["client_resources"], ) -For more advanced information regarding Flower simulation please read the -|flower_how_to_run_simulations_link|_ guide. - Further help ------------ From 0d1a151ead9a9e06990f5a29f975167bcdb9ee16 Mon Sep 17 00:00:00 2001 From: Chong Shen Ng Date: Wed, 20 Nov 2024 10:41:13 +0000 Subject: [PATCH 10/26] Update page name --- doc/source/conf.py | 2 +- ....rst => how-to-upgrade-to-flower-1.13.rst} | 24 +++++++++---------- doc/source/index.rst | 2 +- 3 files changed, 14 insertions(+), 14 deletions(-) rename doc/source/{how-to-upgrade-to-flower-next.rst => how-to-upgrade-to-flower-1.13.rst} (95%) diff --git a/doc/source/conf.py b/doc/source/conf.py index 56324ba113fb..9013858a9636 100644 --- a/doc/source/conf.py +++ b/doc/source/conf.py @@ -19,7 +19,6 @@ import sys from git import Repo -from sphinx.application import ConfigError # Configuration file for the Sphinx documentation builder. # @@ -231,6 +230,7 @@ def find_test_modules(package_path): "logging": "how-to-configure-logging.html", "ssl-enabled-connections": "how-to-enable-ssl-connections.html", "upgrade-to-flower-1.0": "how-to-upgrade-to-flower-1.0.html", + "how-to-upgrade-to-flower-next": "how-to-upgrade-to-flower-1.13.html", # Restructuring: explanations "evaluation": "explanation-federated-evaluation.html", "differential-privacy-wrappers": "explanation-differential-privacy.html", diff --git a/doc/source/how-to-upgrade-to-flower-next.rst b/doc/source/how-to-upgrade-to-flower-1.13.rst similarity index 95% rename from doc/source/how-to-upgrade-to-flower-next.rst rename to doc/source/how-to-upgrade-to-flower-1.13.rst index 9ac94e57988c..ac3996fb05b1 100644 --- a/doc/source/how-to-upgrade-to-flower-next.rst +++ b/doc/source/how-to-upgrade-to-flower-1.13.rst @@ -1,16 +1,16 @@ -Upgrade to Flower Next +Upgrade to Flower 1.13 ====================== -Welcome to the migration guide for updating Flower to Flower Next! Whether you're a +Welcome to the migration guide for updating Flower to Flower 1.13! Whether you're a seasoned user or just getting started, this guide will help you smoothly transition your -existing setup to take advantage of the latest features and improvements in Flower Next, +existing setup to take advantage of the latest features and improvements in Flower 1.13, starting from version 1.8. .. note:: This guide shows how to reuse pre-``1.8`` Flower code with minimum code changes by - using the *compatibility layer* in Flower Next. In another guide, we will show how - to run Flower Next end-to-end with pure Flower Next APIs. + using the *compatibility layer* in Flower 1.13. In another guide, we will show how + to run Flower 1.13 end-to-end with pure Flower 1.13 APIs. Let's dive in! @@ -74,13 +74,13 @@ Let's dive in! Install update -------------- -Here's how to update an existing installation of Flower to Flower Next with ``pip``: +Here's how to update an existing installation of Flower to Flower 1.13 with ``pip``: .. code-block:: bash $ python -m pip install -U flwr -or if you need Flower Next with simulation: +or if you need Flower 1.13 with simulation: .. code-block:: bash @@ -107,14 +107,14 @@ or ``pyproject.toml``: Required changes ---------------- -In Flower Next, the *infrastructure* and *application layers* have been decoupled. +In Flower 1.13, the *infrastructure* and *application layers* have been decoupled. Instead of starting a client in code via ``start_client()``, you create a |clientapp_link|_ and start it via the command line. Instead of starting a server in code via ``start_server()``, you create a |serverapp_link|_ and start it via the command line. The long-running components of server and client are called `SuperLink` and `SuperNode`, for more details please see the |flower_architecture_link|_ . The following non-breaking changes that require manual updates and allow you to run your project both -in the traditional way and in the Flower Next way: +in the traditional way and in the Flower 1.13 way: |clientapp_link|_ ~~~~~~~~~~~~~~~~~ @@ -445,15 +445,15 @@ Further help ------------ Some official `Flower code examples `_ are already -updated to Flower Next so they can serve as a reference for using the Flower Next API. +updated to Flower 1.13 so they can serve as a reference for using the Flower 1.13 API. If there are further questions, `join the Flower Slack `_ and use the channel ``#questions``. You can also `participate in Flower Discuss `_ where you can find us answering questions, or share and -learn from others about migrating to Flower Next. +learn from others about migrating to Flower 1.13. .. admonition:: Important - As we continuously enhance Flower Next at a rapid pace, we'll be periodically + As we continuously enhance Flower 1.13 at a rapid pace, we'll be periodically updating this guide. Please feel free to share any feedback with us! .. diff --git a/doc/source/index.rst b/doc/source/index.rst index dd8e58534674..4397062a392b 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -101,7 +101,7 @@ Problem-oriented how-to guides show step-by-step how to achieve a specific goal. how-to-implement-fedbn docker/index how-to-upgrade-to-flower-1.0 - how-to-upgrade-to-flower-next + how-to-upgrade-to-flower-1.13 Explanations ~~~~~~~~~~~~ From 801d807ceaafbc5818e8a93237306b41d060dd23 Mon Sep 17 00:00:00 2001 From: Chong Shen Ng Date: Wed, 20 Nov 2024 10:49:34 +0000 Subject: [PATCH 11/26] Update text --- doc/source/how-to-upgrade-to-flower-1.13.rst | 23 ++++++++++---------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/doc/source/how-to-upgrade-to-flower-1.13.rst b/doc/source/how-to-upgrade-to-flower-1.13.rst index ac3996fb05b1..3618b4b762b6 100644 --- a/doc/source/how-to-upgrade-to-flower-1.13.rst +++ b/doc/source/how-to-upgrade-to-flower-1.13.rst @@ -8,8 +8,8 @@ starting from version 1.8. .. note:: - This guide shows how to reuse pre-``1.8`` Flower code with minimum code changes by - using the *compatibility layer* in Flower 1.13. In another guide, we will show how + This guide shows how to reuse pre-``1.13`` Flower code with minimum code changes. + In another guide, we will show how to run Flower 1.13 end-to-end with pure Flower 1.13 APIs. Let's dive in! @@ -89,30 +89,31 @@ or if you need Flower 1.13 with simulation: .. code-block:: # Without simulation support - flwr>=1.8,<2.0 + flwr>=1.13,<2.0 # With simulation support - flwr[simulation]>=1.8, <2.0 + flwr[simulation]>=1.13, <2.0 or ``pyproject.toml``: .. code-block:: toml # Without simulation support - dependencies = ["flwr>=1.8,2.0"] + dependencies = ["flwr>=1.13,2.0"] # With simulation support - dependencies = ["flwr[simulation]>=1.8,2.0"] + dependencies = ["flwr[simulation]>=1.13,2.0"] Required changes ---------------- -In Flower 1.13, the *infrastructure* and *application layers* have been decoupled. +From Flower 1.8, the *infrastructure* and *application layers* have been decoupled. +In Flower 1.13, we further enforce this separation. Instead of starting a client in code via ``start_client()``, you create a -|clientapp_link|_ and start it via the command line. Instead of starting a server in -code via ``start_server()``, you create a |serverapp_link|_ and start it via the command -line. The long-running components of server and client are called `SuperLink` and -`SuperNode`, for more details please see the |flower_architecture_link|_ . The following +|clientapp_link|_. Instead of starting a server in +code via ``start_server()``, you create a |serverapp_link|_. Both ``ClientApp`` and ``ServerApp`` +are started by the long-running components of the server and client: the `SuperNode` and +`SuperLink`, respectively. For more details please see the |flower_architecture_link|_ . The following non-breaking changes that require manual updates and allow you to run your project both in the traditional way and in the Flower 1.13 way: From d2a2a945bda726926f22263573cab744109b1ffa Mon Sep 17 00:00:00 2001 From: Chong Shen Ng Date: Wed, 20 Nov 2024 11:13:02 +0000 Subject: [PATCH 12/26] Format --- doc/source/how-to-upgrade-to-flower-1.13.rst | 24 ++++++++++---------- 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/doc/source/how-to-upgrade-to-flower-1.13.rst b/doc/source/how-to-upgrade-to-flower-1.13.rst index 3618b4b762b6..3673603ea2f6 100644 --- a/doc/source/how-to-upgrade-to-flower-1.13.rst +++ b/doc/source/how-to-upgrade-to-flower-1.13.rst @@ -8,9 +8,9 @@ starting from version 1.8. .. note:: - This guide shows how to reuse pre-``1.13`` Flower code with minimum code changes. - In another guide, we will show how - to run Flower 1.13 end-to-end with pure Flower 1.13 APIs. + This guide shows how to reuse pre-``1.13`` Flower code with minimum code changes. In + another guide, we will show how to run Flower 1.13 end-to-end with pure Flower 1.13 + APIs. Let's dive in! @@ -107,15 +107,15 @@ or ``pyproject.toml``: Required changes ---------------- -From Flower 1.8, the *infrastructure* and *application layers* have been decoupled. -In Flower 1.13, we further enforce this separation. -Instead of starting a client in code via ``start_client()``, you create a -|clientapp_link|_. Instead of starting a server in -code via ``start_server()``, you create a |serverapp_link|_. Both ``ClientApp`` and ``ServerApp`` -are started by the long-running components of the server and client: the `SuperNode` and -`SuperLink`, respectively. For more details please see the |flower_architecture_link|_ . The following -non-breaking changes that require manual updates and allow you to run your project both -in the traditional way and in the Flower 1.13 way: +From Flower 1.8, the *infrastructure* and *application layers* have been decoupled. In +Flower 1.13, we further enforce this separation. Instead of starting a client in code +via ``start_client()``, you create a |clientapp_link|_. Instead of starting a server in +code via ``start_server()``, you create a |serverapp_link|_. Both ``ClientApp`` and +``ServerApp`` are started by the long-running components of the server and client: the +`SuperNode` and `SuperLink`, respectively. For more details please see the +|flower_architecture_link|_ . The following non-breaking changes that require manual +updates and allow you to run your project both in the traditional way and in the Flower +1.13 way: |clientapp_link|_ ~~~~~~~~~~~~~~~~~ From 7684eff24c54cda92e1b2056b34869369ad00303 Mon Sep 17 00:00:00 2001 From: Chong Shen Ng Date: Wed, 20 Nov 2024 11:51:25 +0000 Subject: [PATCH 13/26] Update text, minor fixes --- doc/source/how-to-upgrade-to-flower-1.13.rst | 88 +++++++++++--------- 1 file changed, 47 insertions(+), 41 deletions(-) diff --git a/doc/source/how-to-upgrade-to-flower-1.13.rst b/doc/source/how-to-upgrade-to-flower-1.13.rst index 3673603ea2f6..a107300a0bd4 100644 --- a/doc/source/how-to-upgrade-to-flower-1.13.rst +++ b/doc/source/how-to-upgrade-to-flower-1.13.rst @@ -19,9 +19,9 @@ Let's dive in! - https://stackoverflow.com/q/71651598 - https://github.com/jgm/pandoc/issues/3973#issuecomment-337087394 -.. |clientapp_link| replace:: ``ClientApp()`` +.. |clientapp_link| replace:: ``ClientApp`` -.. |serverapp_link| replace:: ``ServerApp()`` +.. |serverapp_link| replace:: ``ServerApp`` .. |startclient_link| replace:: ``start_client()`` @@ -126,9 +126,9 @@ updates and allow you to run your project both in the traditional way and in the .. code-block:: python :emphasize-lines: 2,6 - # Flower v1.11+ + # Flower v1.10+ def client_fn(context: flwr.common.Context): - return flwr.client.FlowerClient().to_client() + return FlowerClient().to_client() app = flwr.client.ClientApp( @@ -136,9 +136,9 @@ updates and allow you to run your project both in the traditional way and in the ) - # Flower v1.8 - v1.10 + # Flower v1.8 - v1.9 def client_fn(cid: str): - return flwr.client.FlowerClient().to_client() + return FlowerClient().to_client() app = flwr.client.ClientApp( @@ -149,7 +149,7 @@ updates and allow you to run your project both in the traditional way and in the if __name__ == "__main__": flwr.client.start_client( server_address="127.0.0.1:8080", - client=flwr.client.FlowerClient().to_client(), + client=FlowerClient().to_client(), ) |serverapp_link|_ @@ -161,18 +161,18 @@ updates and allow you to run your project both in the traditional way and in the .. code-block:: python :emphasize-lines: 2,8 - # Flower v1.11+ + # Flower v1.10+ def server_fn(context: flwr.common.Context): strategy = flwr.server.strategy.FedAvg() config = flwr.server.ServerConfig() - return flwr.server.ServerAppComponents(strategy=strategy, config=config) + return flwr.server.ServerAppComponents(config=config, strategy=strategy) app = flwr.server.ServerApp( server_fn=server_fn, ) - # Flower v1.8 - v1.11 + # Flower v1.8 - v1.9 app = flwr.server.ServerApp( config=config, strategy=strategy, @@ -189,10 +189,17 @@ updates and allow you to run your project both in the traditional way and in the Deployment ~~~~~~~~~~ -- Run the ``SuperLink`` using |flowernext_superlink_link|_ before running, in sequence, - |flowernext_supernode_link|_ (2x). +- In CLI, start the SuperLink using |flowernext_superlink_link|_. Then, start two + SuperNodes using |flowernext_supernode_link|_ (2x). There is no need to run + ``flower-server-app`` and ``flower-client-app``, or execute ``client.py`` and + ``server.py`` as Python scripts. - Here's an example to start the server without HTTPS (only for prototyping): +.. tip:: + + For a comprehensive walk-through on how to run Flower deployment using Docker, + please refer to the :doc:`docker/index` guide. + .. code-block:: bash :emphasize-lines: 2,5,12 @@ -203,15 +210,15 @@ Deployment $ flower-supernode \ --insecure \ --superlink 127.0.0.1:9092 \ - --node-config "..." \ - --supernode-address 127.0.0.1:9094 + --supernode-address 127.0.0.1:9094 \ + # In another terminal window, start another long-running SuperNode (at least 2 SuperNodes are required) $ flower-supernode \ --insecure \ --superlink 127.0.0.1:9092 \ - --node-config "..." \ - --supernode-address 127.0.0.1:9095 + --supernode-address 127.0.0.1:9095 \ + - Here's another example to start with HTTPS. Use the ``--ssl-ca-certfile``, ``--ssl-certfile``, and ``--ssl-keyfile`` command line options to pass paths to (CA @@ -229,16 +236,16 @@ Deployment # In a new terminal window, start a long-running SuperNode $ flower-supernode \ --superlink 127.0.0.1:9092 \ - --node-config "..." \ --supernode-address 127.0.0.1:9094 \ - --root-certificates + --root-certificates \ + # In another terminal window, start another long-running SuperNode (at least 2 SuperNodes are required) $ flower-supernode \ --superlink 127.0.0.1:9092 \ - --node-config "..." \ --supernode-address 127.0.0.1:9095 \ - --root-certificates + --root-certificates \ + Simulation in CLI ~~~~~~~~~~~~~~~~~ @@ -248,11 +255,11 @@ respectively. There is no need to use |startsim_link|_ anymore. Here's an exampl .. tip:: - For more advanced information regarding Flower simulation please read the - |flower_how_to_run_simulations_link|_ guide. + For a comprehensive walk-through on how to run Flower simulations please refer to + the |flower_how_to_run_simulations_link|_ guide. .. code-block:: python - :emphasize-lines: 9,13,18,25 + :emphasize-lines: 8,12,17,23 # Regular Flower client implementation class FlowerClient(NumPyClient): @@ -260,10 +267,9 @@ respectively. There is no need to use |startsim_link|_ anymore. Here's an exampl pass - # Flower v1.11+ - # [file: client_app.py] + # Flower v1.10+ def client_fn(context: flwr.common.Context): - return flwr.client.FlowerClient().to_client() + return FlowerClient().to_client() app = flwr.client.ClientApp( @@ -271,7 +277,6 @@ respectively. There is no need to use |startsim_link|_ anymore. Here's an exampl ) - # [file: server_app.py] def server_fn(context: flwr.common.Context): strategy = flwr.server.strategy.FedAvg(...) config = flwr.server.ServerConfig(...) @@ -283,7 +288,7 @@ respectively. There is no need to use |startsim_link|_ anymore. Here's an exampl ) - # Flower v1.8 - v1.10 + # Flower v1.8 - v1.9 def client_fn(cid: str): return FlowerClient().to_client() @@ -297,6 +302,7 @@ respectively. There is no need to use |startsim_link|_ anymore. Here's an exampl strategy=strategy, ) + # Flower v1.7 if __name__ == "__main__": hist = flwr.simulation.start_simulation( @@ -306,12 +312,12 @@ respectively. There is no need to use |startsim_link|_ anymore. Here's an exampl Depending on your Flower version, you can run your simulation as follows: -- for Flower versions 1.11 and onwards, run ``flwr run`` in CLI. -- for Flower versions between 1.8 to 1.10, run |flower_simulation_link|_ in CLI and +- For Flower versions 1.11 and onwards, run ``flwr run`` in CLI. +- For Flower versions between 1.8 to 1.10, run |flower_simulation_link|_ in CLI and point to the ``server_app`` / ``client_app`` object in the code instead of executing the Python script. In the code snippet below, there is an example (assuming the ``server_app`` and ``client_app`` objects are in a ``sim.py`` module). -- for Flower versions before 1.8, run the Python script directly. +- For Flower versions before 1.8, run the Python script directly. .. code-block:: bash :emphasize-lines: 2 @@ -332,19 +338,19 @@ Depending on your Flower version, you can run your simulation as follows: Depending on your Flower version, you can also define the default resources as follows: -- for Flower versions 1.11 and onwards, you can edit your pyproject.toml file and then - run ``flwr run`` in CLI as shown in the example below. -- for Flower versions between 1.8 to 1.10, you can adjust the resources for each +- For Flower versions 1.11 and onwards, you can edit your ``pyproject.toml`` file and + then run ``flwr run`` in CLI as shown in the example below. +- For Flower versions between 1.8 to 1.10, you can adjust the resources for each |clientapp_link|_ using the ``--backend-config`` command line argument instead of setting the ``client_resources`` argument in |startsim_link|_. -- for Flower versions before 1.8, you need to run |startsim_link|_ and pass a dictionary +- For Flower versions before 1.8, you need to run |startsim_link|_ and pass a dictionary of the required resources to the ``client_resources`` argument. .. code-block:: bash :emphasize-lines: 2,8 # Flower v.1.11+ - # pyproject.toml + # [file: pyproject.toml] [tool.flwr.federations.local-sim-gpu] options.num-supernodes = 10 options.backend.client-resources.num-cpus = 2 @@ -371,11 +377,11 @@ Simulation in a Notebook To run your simulation from within a notebook, please consider the following examples depending on your Flower version: -- for Flower versions 1.11 and onwards, you need to run |runsim_link|_ in your notebook +- For Flower versions 1.11 and onwards, you need to run |runsim_link|_ in your notebook instead of |startsim_link|_. -- for Flower versions between 1.8 to 1.10, you need to run |runsim_link|_ in your +- For Flower versions between 1.8 to 1.10, you need to run |runsim_link|_ in your notebook instead of |startsim_link|_ and configure the resources. -- for Flower versions before 1.8, you need to run |startsim_link|_ and pass a dictionary +- For Flower versions before 1.8, you need to run |startsim_link|_ and pass a dictionary of the required resources to the ``client_resources`` argument. .. tip:: @@ -386,9 +392,9 @@ depending on your Flower version: .. code-block:: python :emphasize-lines: 2,6,10,14 - # Flower v1.11+ + # Flower v1.10+ def client_fn(context: flwr.common.Context): - return flwr.client.FlowerClient().to_client() + return FlowerClient().to_client() client_app = flwr.server.ClientApp( From 6bb5c3f41e4b1fe0c681b72e80527f7924686c2b Mon Sep 17 00:00:00 2001 From: Chong Shen Ng Date: Wed, 20 Nov 2024 11:53:20 +0000 Subject: [PATCH 14/26] Small fix --- doc/source/how-to-upgrade-to-flower-1.13.rst | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/doc/source/how-to-upgrade-to-flower-1.13.rst b/doc/source/how-to-upgrade-to-flower-1.13.rst index a107300a0bd4..ac7116bad5d8 100644 --- a/doc/source/how-to-upgrade-to-flower-1.13.rst +++ b/doc/source/how-to-upgrade-to-flower-1.13.rst @@ -460,10 +460,7 @@ learn from others about migrating to Flower 1.13. .. admonition:: Important - As we continuously enhance Flower 1.13 at a rapid pace, we'll be periodically + As we continuously enhance Flower at a rapid pace, we'll be periodically updating this guide. Please feel free to share any feedback with us! -.. - [TODO] Add links to Flower Next 101 and Flower Glossary - Happy migrating! 🚀 From 30fb23de75ef372aa5f166a7f2266c50f9d83a64 Mon Sep 17 00:00:00 2001 From: Chong Shen Ng Date: Wed, 20 Nov 2024 11:54:56 +0000 Subject: [PATCH 15/26] Remove unused sentence --- doc/source/how-to-upgrade-to-flower-1.13.rst | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/doc/source/how-to-upgrade-to-flower-1.13.rst b/doc/source/how-to-upgrade-to-flower-1.13.rst index ac7116bad5d8..c00c3f601d4b 100644 --- a/doc/source/how-to-upgrade-to-flower-1.13.rst +++ b/doc/source/how-to-upgrade-to-flower-1.13.rst @@ -8,9 +8,7 @@ starting from version 1.8. .. note:: - This guide shows how to reuse pre-``1.13`` Flower code with minimum code changes. In - another guide, we will show how to run Flower 1.13 end-to-end with pure Flower 1.13 - APIs. + This guide shows how to reuse pre-``1.13`` Flower code with minimum code changes. Let's dive in! From 2bb5cbfd906087a0d05a9a44dbc9de579c93c035 Mon Sep 17 00:00:00 2001 From: Chong Shen Ng Date: Wed, 20 Nov 2024 12:00:07 +0000 Subject: [PATCH 16/26] Minor text fix --- doc/source/how-to-upgrade-to-flower-1.13.rst | 49 +++++++++++--------- 1 file changed, 26 insertions(+), 23 deletions(-) diff --git a/doc/source/how-to-upgrade-to-flower-1.13.rst b/doc/source/how-to-upgrade-to-flower-1.13.rst index c00c3f601d4b..5ba37a8a2edd 100644 --- a/doc/source/how-to-upgrade-to-flower-1.13.rst +++ b/doc/source/how-to-upgrade-to-flower-1.13.rst @@ -29,13 +29,13 @@ Let's dive in! .. |runsim_link| replace:: ``run_simulation()`` -.. |flowernext_clientapp_link| replace:: ``flower-client-app`` +.. |flower_clientapp_link| replace:: ``flower-client-app`` -.. |flowernext_serverapp_link| replace:: ``flower-server-app`` +.. |flower_serverapp_link| replace:: ``flower-server-app`` -.. |flowernext_superlink_link| replace:: ``flower-superlink`` +.. |flower_superlink_link| replace:: ``flower-superlink`` -.. |flowernext_supernode_link| replace:: ``flower-supernode`` +.. |flower_supernode_link| replace:: ``flower-supernode`` .. |flower_architecture_link| replace:: Flower Architecture @@ -47,17 +47,17 @@ Let's dive in! .. _flower_architecture_link: explanation-flower-architecture.html -.. _flower_how_to_run_simulations_link: how-to-run-simulations.html +.. _flower_clientapp_link: ref-api-cli.html#flower-client-app -.. _flower_simulation_link: ref-api-cli.html#flower-simulation +.. _flower_how_to_run_simulations_link: how-to-run-simulations.html -.. _flowernext_clientapp_link: ref-api-cli.html#flower-client-app +.. _flower_serverapp_link: ref-api-cli.html#flower-server-app -.. _flowernext_serverapp_link: ref-api-cli.html#flower-server-app +.. _flower_simulation_link: ref-api-cli.html#flower-simulation -.. _flowernext_superlink_link: ref-api-cli.html#flower-superlink +.. _flower_superlink_link: ref-api-cli.html#flower-superlink -.. _flowernext_supernode_link: ref-api-cli.html#flower-supernode +.. _flower_supernode_link: ref-api-cli.html#flower-supernode .. _runsim_link: ref-api/flwr.simulation.run_simulation.html @@ -110,10 +110,14 @@ Flower 1.13, we further enforce this separation. Instead of starting a client in via ``start_client()``, you create a |clientapp_link|_. Instead of starting a server in code via ``start_server()``, you create a |serverapp_link|_. Both ``ClientApp`` and ``ServerApp`` are started by the long-running components of the server and client: the -`SuperNode` and `SuperLink`, respectively. For more details please see the -|flower_architecture_link|_ . The following non-breaking changes that require manual -updates and allow you to run your project both in the traditional way and in the Flower -1.13 way: +`SuperNode` and `SuperLink`, respectively. + +.. tip:: + + For more details please see the |flower_architecture_link|_ . + +The following non-breaking changes that require manual updates and allow you to run your +project both in the traditional way and in the Flower 1.13 way: |clientapp_link|_ ~~~~~~~~~~~~~~~~~ @@ -187,10 +191,9 @@ updates and allow you to run your project both in the traditional way and in the Deployment ~~~~~~~~~~ -- In CLI, start the SuperLink using |flowernext_superlink_link|_. Then, start two - SuperNodes using |flowernext_supernode_link|_ (2x). There is no need to run - ``flower-server-app`` and ``flower-client-app``, or execute ``client.py`` and - ``server.py`` as Python scripts. +- In CLI, start the SuperLink using |flower_superlink_link|_. Then, start two SuperNodes + using |flower_supernode_link|_ (2x). There is no need to run ``flower-server-app`` and + ``flower-client-app``, or execute ``client.py`` and ``server.py`` as Python scripts. - Here's an example to start the server without HTTPS (only for prototyping): .. tip:: @@ -253,8 +256,8 @@ respectively. There is no need to use |startsim_link|_ anymore. Here's an exampl .. tip:: - For a comprehensive walk-through on how to run Flower simulations please refer to - the |flower_how_to_run_simulations_link|_ guide. + For a comprehensive guide on how to setup and run Flower simulations please read the + |flower_how_to_run_simulations_link|_ guide. .. code-block:: python :emphasize-lines: 8,12,17,23 @@ -384,7 +387,7 @@ depending on your Flower version: .. tip:: - For more advanced information regarding Flower simulation please read the + For a comprehensive guide on how to setup and run Flower simulations please read the |flower_how_to_run_simulations_link|_ guide. .. code-block:: python @@ -458,7 +461,7 @@ learn from others about migrating to Flower 1.13. .. admonition:: Important - As we continuously enhance Flower at a rapid pace, we'll be periodically - updating this guide. Please feel free to share any feedback with us! + As we continuously enhance Flower at a rapid pace, we'll be periodically updating + this guide. Please feel free to share any feedback with us! Happy migrating! 🚀 From 1430da5e835b74ce38c092b9460f091a572de9ff Mon Sep 17 00:00:00 2001 From: Chong Shen Ng Date: Wed, 20 Nov 2024 12:36:17 +0000 Subject: [PATCH 17/26] Remove mention of next --- doc/source/how-to-upgrade-to-flower-1.0.rst | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/doc/source/how-to-upgrade-to-flower-1.0.rst b/doc/source/how-to-upgrade-to-flower-1.0.rst index ec2bf5ef7479..045303ad750c 100644 --- a/doc/source/how-to-upgrade-to-flower-1.0.rst +++ b/doc/source/how-to-upgrade-to-flower-1.0.rst @@ -4,11 +4,10 @@ Upgrade to Flower 1.0 .. note:: This guide is for users who have already worked with Flower 0.x and want to upgrade - to Flower 1.0. Newer versions of Flower (1.12+) are based on a new architecture - (previously called Flower Next) and not covered in this guide. After upgrading - Flower 0.x projects to Flower 1.0, please refer to :doc:`Upgrade to Flower Next - ` to make your project compatible with the lastest - version of Flower. + to Flower 1.0. Newer versions of Flower (1.12+) are based on a new architecture and + not covered in this guide. After upgrading Flower 0.x projects to Flower 1.0, please + refer to :doc:`Upgrade to Flower 1.13 ` to make your + project compatible with the lastest version of Flower. Flower 1.0 is here. Along with new features, Flower 1.0 provides a stable foundation for future growth. Compared to Flower 0.19 (and other 0.x series releases), there are a few From f50ad77476d19e5f50fc3eefd71669df26bafe62 Mon Sep 17 00:00:00 2001 From: "Daniel J. Beutel" Date: Wed, 20 Nov 2024 14:10:34 +0100 Subject: [PATCH 18/26] Edit --- doc/source/how-to-upgrade-to-flower-1.0.rst | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/doc/source/how-to-upgrade-to-flower-1.0.rst b/doc/source/how-to-upgrade-to-flower-1.0.rst index 045303ad750c..7643c347ac0f 100644 --- a/doc/source/how-to-upgrade-to-flower-1.0.rst +++ b/doc/source/how-to-upgrade-to-flower-1.0.rst @@ -4,10 +4,11 @@ Upgrade to Flower 1.0 .. note:: This guide is for users who have already worked with Flower 0.x and want to upgrade - to Flower 1.0. Newer versions of Flower (1.12+) are based on a new architecture and - not covered in this guide. After upgrading Flower 0.x projects to Flower 1.0, please - refer to :doc:`Upgrade to Flower 1.13 ` to make your - project compatible with the lastest version of Flower. + to Flower 1.0. Newer versions of Flower (1.13 and later) are based on a new + architecture and not covered in this guide. After upgrading Flower 0.x projects to + Flower 1.0, please refer to :doc:`Upgrade to Flower 1.13 + ` to make your project compatible with the lastest + version of Flower. Flower 1.0 is here. Along with new features, Flower 1.0 provides a stable foundation for future growth. Compared to Flower 0.19 (and other 0.x series releases), there are a few From 88ce30b1585de413290b4ea358b11a6caa71521c Mon Sep 17 00:00:00 2001 From: "Daniel J. Beutel" Date: Wed, 20 Nov 2024 14:12:14 +0100 Subject: [PATCH 19/26] Edit --- doc/source/how-to-upgrade-to-flower-1.13.rst | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/doc/source/how-to-upgrade-to-flower-1.13.rst b/doc/source/how-to-upgrade-to-flower-1.13.rst index 5ba37a8a2edd..10c688ebf0c9 100644 --- a/doc/source/how-to-upgrade-to-flower-1.13.rst +++ b/doc/source/how-to-upgrade-to-flower-1.13.rst @@ -3,12 +3,12 @@ Upgrade to Flower 1.13 Welcome to the migration guide for updating Flower to Flower 1.13! Whether you're a seasoned user or just getting started, this guide will help you smoothly transition your -existing setup to take advantage of the latest features and improvements in Flower 1.13, -starting from version 1.8. +existing setup to take advantage of the latest features and improvements in Flower 1.13. .. note:: - This guide shows how to reuse pre-``1.13`` Flower code with minimum code changes. + This guide shows how to make pre-``1.13`` Flower code compatible with Flower 1.13 + (and later) with only minimal code changes. Let's dive in! @@ -84,6 +84,8 @@ or if you need Flower 1.13 with simulation: $ python -m pip install -U "flwr[simulation]" +Ensure you set the following version constraint in your ``requirements.txt`` + .. code-block:: # Without simulation support @@ -97,10 +99,14 @@ or ``pyproject.toml``: .. code-block:: toml # Without simulation support - dependencies = ["flwr>=1.13,2.0"] + dependencies = [ + "flwr>=1.13,2.0", + ] # With simulation support - dependencies = ["flwr[simulation]>=1.13,2.0"] + dependencies = [ + "flwr[simulation]>=1.13,2.0", + ] Required changes ---------------- From 6b0ecf8675967458edbeea4aceca9a3dcd750315 Mon Sep 17 00:00:00 2001 From: "Daniel J. Beutel" Date: Wed, 20 Nov 2024 14:21:48 +0100 Subject: [PATCH 20/26] Update required changes section --- doc/source/how-to-upgrade-to-flower-1.13.rst | 90 +++++++++++--------- 1 file changed, 50 insertions(+), 40 deletions(-) diff --git a/doc/source/how-to-upgrade-to-flower-1.13.rst b/doc/source/how-to-upgrade-to-flower-1.13.rst index 10c688ebf0c9..ca5c211b33a5 100644 --- a/doc/source/how-to-upgrade-to-flower-1.13.rst +++ b/doc/source/how-to-upgrade-to-flower-1.13.rst @@ -111,19 +111,24 @@ or ``pyproject.toml``: Required changes ---------------- -From Flower 1.8, the *infrastructure* and *application layers* have been decoupled. In -Flower 1.13, we further enforce this separation. Instead of starting a client in code -via ``start_client()``, you create a |clientapp_link|_. Instead of starting a server in -code via ``start_server()``, you create a |serverapp_link|_. Both ``ClientApp`` and -``ServerApp`` are started by the long-running components of the server and client: the -`SuperNode` and `SuperLink`, respectively. +Starting with Flower 1.8, the *infrastructure* and *application layers* have been +decoupled. Flower 1.13 enforces this separation further. Among other things, this allows +you to run the exact same code in a simulation as in a real deployment. + +Instead of starting a client in code via ``start_client()``, you create a +|clientapp_link|_. Instead of starting a server in code via ``start_server()``, you +create a |serverapp_link|_. Both ``ClientApp`` and ``ServerApp`` are started by the +long-running components of the server and client: the `SuperLink` and `SuperNode`, +respectively. .. tip:: - For more details please see the |flower_architecture_link|_ . + For more details on SuperLink and SuperNode, please see the + |flower_architecture_link|_ . -The following non-breaking changes that require manual updates and allow you to run your -project both in the traditional way and in the Flower 1.13 way: +The following non-breaking changes require manual updates and allow you to run your +project both in the traditional (now deprecated) way and in the new (recommended) Flower +1.13 way: |clientapp_link|_ ~~~~~~~~~~~~~~~~~ @@ -132,30 +137,30 @@ project both in the traditional way and in the Flower 1.13 way: |startclient_link|_. Here's an example: .. code-block:: python - :emphasize-lines: 2,6 + :emphasize-lines: 6,10 - # Flower v1.10+ - def client_fn(context: flwr.common.Context): - return FlowerClient().to_client() + from flwr.client import ClientApp, start_client + from flwr.common import Context - app = flwr.client.ClientApp( - client_fn=client_fn, - ) + # Flower 1.10 and later (recommended) + def client_fn(context: Context): + return FlowerClient().to_client() - # Flower v1.8 - v1.9 - def client_fn(cid: str): - return FlowerClient().to_client() + app = ClientApp(client_fn=client_fn) - app = flwr.client.ClientApp( - client_fn=client_fn, - ) + # # Flower 1.8 - 1.9 (deprecated, no longer supported) + # def client_fn(cid: str): + # return FlowerClient().to_client() + # + # app = ClientApp(client_fn=client_fn) - # Flower v1.7 + + # Flower 1.7 (deprecated, only for backwards-compatibility) if __name__ == "__main__": - flwr.client.start_client( + start_client( server_address="127.0.0.1:8080", client=FlowerClient().to_client(), ) @@ -167,28 +172,33 @@ project both in the traditional way and in the Flower 1.13 way: |startserver_link|_. Here's an example: .. code-block:: python - :emphasize-lines: 2,8 + :emphasize-lines: 7,13 - # Flower v1.10+ - def server_fn(context: flwr.common.Context): - strategy = flwr.server.strategy.FedAvg() - config = flwr.server.ServerConfig() - return flwr.server.ServerAppComponents(config=config, strategy=strategy) + from flwr.common import Context + from flwr.server import ServerApp, ServerAppComponents, ServerConfig, start_server + from flwr.server.strategy import FedAvg - app = flwr.server.ServerApp( - server_fn=server_fn, - ) + # Flower 1.10 and later (recommended) + def server_fn(context: Context): + strategy = FedAvg() + config = ServerConfig() + return ServerAppComponents(config=config, strategy=strategy) - # Flower v1.8 - v1.9 - app = flwr.server.ServerApp( - config=config, - strategy=strategy, - ) - # Flower v1.7 + app = ServerApp(server_fn=server_fn) + + + # # Flower 1.8 - 1.9 (deprecated, no longer supported) + # app = flwr.server.ServerApp( + # config=config, + # strategy=strategy, + # ) + + + # Flower 1.7 (deprecated, only for backwards-compatibility) if __name__ == "__main__": - flwr.server.start_server( + start_server( server_address="0.0.0.0:8080", config=config, strategy=strategy, From 9dd544c883b1849d8b436c6a84aef852ebb9a1d2 Mon Sep 17 00:00:00 2001 From: "Daniel J. Beutel" Date: Wed, 20 Nov 2024 14:29:07 +0100 Subject: [PATCH 21/26] Edit --- doc/source/how-to-upgrade-to-flower-1.13.rst | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/doc/source/how-to-upgrade-to-flower-1.13.rst b/doc/source/how-to-upgrade-to-flower-1.13.rst index ca5c211b33a5..60e631ff6d2b 100644 --- a/doc/source/how-to-upgrade-to-flower-1.13.rst +++ b/doc/source/how-to-upgrade-to-flower-1.13.rst @@ -207,20 +207,20 @@ project both in the traditional (now deprecated) way and in the new (recommended Deployment ~~~~~~~~~~ -- In CLI, start the SuperLink using |flower_superlink_link|_. Then, start two SuperNodes - using |flower_supernode_link|_ (2x). There is no need to run ``flower-server-app`` and - ``flower-client-app``, or execute ``client.py`` and ``server.py`` as Python scripts. -- Here's an example to start the server without HTTPS (only for prototyping): +- In a terminal window, start the SuperLink using |flower_superlink_link|_. Then, in two + additional terminal windows, start two SuperNodes using |flower_supernode_link|_ (2x). + There is no need to directly run ``client.py`` and ``server.py`` as Python scripts. +- Here's an example to start the server without HTTPS (insecure mode, only for prototyping): .. tip:: - For a comprehensive walk-through on how to run Flower deployment using Docker, + For a comprehensive walk-through on how to deploy Flower using Docker, please refer to the :doc:`docker/index` guide. .. code-block:: bash :emphasize-lines: 2,5,12 - # Start a Superlink + # Start a SuperLink $ flower-superlink --insecure # In a new terminal window, start a long-running SuperNode @@ -244,7 +244,7 @@ Deployment .. code-block:: bash :emphasize-lines: 2,8,15 - # Start a secure Superlink + # Start a secure SuperLink $ flower-superlink \ --ssl-ca-certfile \ --ssl-certfile \ From 336cafded27717dfd58981ce94926d5a837dbb47 Mon Sep 17 00:00:00 2001 From: "Daniel J. Beutel" Date: Wed, 20 Nov 2024 14:42:38 +0100 Subject: [PATCH 22/26] Edit --- doc/source/how-to-upgrade-to-flower-1.13.rst | 158 ++++++++++--------- 1 file changed, 82 insertions(+), 76 deletions(-) diff --git a/doc/source/how-to-upgrade-to-flower-1.13.rst b/doc/source/how-to-upgrade-to-flower-1.13.rst index 60e631ff6d2b..912a465e01a8 100644 --- a/doc/source/how-to-upgrade-to-flower-1.13.rst +++ b/doc/source/how-to-upgrade-to-flower-1.13.rst @@ -155,6 +155,7 @@ project both in the traditional (now deprecated) way and in the new (recommended # def client_fn(cid: str): # return FlowerClient().to_client() # + # # app = ClientApp(client_fn=client_fn) @@ -210,12 +211,13 @@ Deployment - In a terminal window, start the SuperLink using |flower_superlink_link|_. Then, in two additional terminal windows, start two SuperNodes using |flower_supernode_link|_ (2x). There is no need to directly run ``client.py`` and ``server.py`` as Python scripts. -- Here's an example to start the server without HTTPS (insecure mode, only for prototyping): +- Here's an example to start the server without HTTPS (insecure mode, only for + prototyping): .. tip:: - For a comprehensive walk-through on how to deploy Flower using Docker, - please refer to the :doc:`docker/index` guide. + For a comprehensive walk-through on how to deploy Flower using Docker, please refer + to the :doc:`docker/index` guide. .. code-block:: bash :emphasize-lines: 2,5,12 @@ -237,9 +239,9 @@ Deployment --supernode-address 127.0.0.1:9095 \ -- Here's another example to start with HTTPS. Use the ``--ssl-ca-certfile``, - ``--ssl-certfile``, and ``--ssl-keyfile`` command line options to pass paths to (CA - certificate, server certificate, and server private key). +- Here's another example to start both SuperLink and SuperNodes with HTTPS. Use the + ``--ssl-ca-certfile``, ``--ssl-certfile``, and ``--ssl-keyfile`` command line options + to pass paths to (CA certificate, server certificate, and server private key). .. code-block:: bash :emphasize-lines: 2,8,15 @@ -276,7 +278,14 @@ respectively. There is no need to use |startsim_link|_ anymore. Here's an exampl |flower_how_to_run_simulations_link|_ guide. .. code-block:: python - :emphasize-lines: 8,12,17,23 + :emphasize-lines: 9,15,19,22,28 + + from flwr.client import ClientApp + from flwr.common import Context + from flwr.server import ServerApp, ServerAppComponents, ServerConfig + from flwr.server.strategy import FedAvg + from flwr.simulation import start_simulation + # Regular Flower client implementation class FlowerClient(NumPyClient): @@ -284,45 +293,40 @@ respectively. There is no need to use |startsim_link|_ anymore. Here's an exampl pass - # Flower v1.10+ - def client_fn(context: flwr.common.Context): + # Flower 1.10 and later (recommended) + def client_fn(context: Context): return FlowerClient().to_client() - app = flwr.client.ClientApp( - client_fn=client_fn, - ) - - - def server_fn(context: flwr.common.Context): - strategy = flwr.server.strategy.FedAvg(...) - config = flwr.server.ServerConfig(...) - return flwr.server.ServerAppComponents(strategy=strategy, config=config) + app = ClientApp(client_fn=client_fn) - server_app = flwr.server.ServerApp( - server_fn=server_fn, - ) + def server_fn(context: Context): + strategy = FedAvg(...) + config = ServerConfig(...) + return ServerAppComponents(strategy=strategy, config=config) - # Flower v1.8 - v1.9 - def client_fn(cid: str): - return FlowerClient().to_client() + server_app = ServerApp(server_fn=server_fn) - client_app = flwr.client.ClientApp( - client_fn=client_fn, - ) - - server_app = flwr.server.ServerApp( - config=config, - strategy=strategy, - ) + # # Flower 1.8 - 1.9 (deprecated, no longer supported) + # def client_fn(cid: str): + # return FlowerClient().to_client() + # + # + # client_app = ClientApp(client_fn=client_fn) + # + # + # server_app = ServerApp( + # config=config, + # strategy=strategy, + # ) - # Flower v1.7 + # Flower 1.7 (deprecated, only for backwards-compatibility) if __name__ == "__main__": - hist = flwr.simulation.start_simulation( + hist = start_simulation( num_clients=10, # ... ) @@ -339,18 +343,18 @@ Depending on your Flower version, you can run your simulation as follows: .. code-block:: bash :emphasize-lines: 2 - # Flower v1.11+ + # Flower 1.11 and later (recommended) $ flwr run - # Flower v1.8 - v1.10 + # Flower 1.8 - 1.10 (deprecated, no longer supported) $ flower-simulation \ --server-app=sim:server_app \ --client-app=sim:client_app \ --num-supernodes=10 - # Flower v1.7 + # Flower 1.7 (deprecated) $ python sim.py Depending on your Flower version, you can also define the default resources as follows: @@ -366,7 +370,7 @@ Depending on your Flower version, you can also define the default resources as f .. code-block:: bash :emphasize-lines: 2,8 - # Flower v.1.11+ + # Flower 1.11 and later (recommended) # [file: pyproject.toml] [tool.flwr.federations.local-sim-gpu] options.num-supernodes = 10 @@ -375,16 +379,18 @@ Depending on your Flower version, you can also define the default resources as f $ flwr run - # Flower v1.8 - v1.10 + # Flower 1.8 - 1.10 (deprecated, no longer supported) $ flower-simulation \ --client-app=sim:client_app \ --server-app=sim:server_app \ --num-supernodes=10 \ --backend-config='{"client_resources": {"num_cpus": 2, "num_gpus": 0.25}}' - # Flower v1.7 (in `sim.py`) +.. code-block:: python + + # Flower 1.7 (in `sim.py`, deprecated) if __name__ == "__main__": - hist = flwr.simulation.start_simulation( + hist = start_simulation( num_clients=10, client_resources={"num_cpus": 2, "num_gpus": 0.25}, ... ) @@ -409,55 +415,55 @@ depending on your Flower version: .. code-block:: python :emphasize-lines: 2,6,10,14 - # Flower v1.10+ - def client_fn(context: flwr.common.Context): - return FlowerClient().to_client() - - - client_app = flwr.server.ClientApp( - client_fn=client_fn, - ) - - server_app = flwr.server.ServerApp( - server_fn=server_fn, - ) - - flwr.simulation.run_simulation( - server_app=server_app, - client_app=client_app, - ) - - - # Flower v1.8 - v1.10 - NUM_CLIENTS = 10 # Replace by any integer greater than zero - backend_config = {"client_resources": {"num_cpus": 2, "num_gpus": 0.25}} + from flwr.client import ClientApp + from flwr.common import Context + from flwr.server import ServerApp + from flwr.simulation import run_simulation, start_simulation - def client_fn(cid: str): - # ... + # Flower 1.10 and later (recommended) + def client_fn(context: Context): return FlowerClient().to_client() - client_app = flwr.client.ClientApp( - client_fn=client_fn, - ) + client_app = ClientApp(client_fn=client_fn) - server_app = flwr.server.ServerApp( - config=config, - strategy=strategy, - ) + server_app = ServerApp(server_fn=server_fn) - flwr.simulation.run_simulation( + run_simulation( server_app=server_app, client_app=client_app, - num_supernodes=NUM_CLIENTS, - backend_config=backend_config, ) + + # # Flower v1.8 - v1.10 (deprecated, no longer supported) + # NUM_CLIENTS = 10 # Replace by any integer greater than zero + # backend_config = {"client_resources": {"num_cpus": 2, "num_gpus": 0.25}} + # + # + # def client_fn(cid: str): + # # ... + # return FlowerClient().to_client() + # + # + # client_app = ClientApp(client_fn=client_fn) + # + # server_app = ServerApp( + # config=config, + # strategy=strategy, + # ) + # + # run_simulation( + # server_app=server_app, + # client_app=client_app, + # num_supernodes=NUM_CLIENTS, + # backend_config=backend_config, + # ) + # Flower v1.7 NUM_CLIENTS = 10 # Replace by any integer greater than zero backend_config = {"client_resources": {"num_cpus": 2, "num_gpus": 0.25}} - flwr.simulation.start_simulation( + start_simulation( client_fn=client_fn, num_clients=NUM_CLIENTS, config=config, From 229e987194b6f0251e5b2d7685b4b42a7299ddc6 Mon Sep 17 00:00:00 2001 From: Chong Shen Ng Date: Wed, 20 Nov 2024 13:44:25 +0000 Subject: [PATCH 23/26] Fix arg name --- doc/source/how-to-upgrade-to-flower-1.13.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/doc/source/how-to-upgrade-to-flower-1.13.rst b/doc/source/how-to-upgrade-to-flower-1.13.rst index 912a465e01a8..767c2ba7cff5 100644 --- a/doc/source/how-to-upgrade-to-flower-1.13.rst +++ b/doc/source/how-to-upgrade-to-flower-1.13.rst @@ -229,14 +229,14 @@ Deployment $ flower-supernode \ --insecure \ --superlink 127.0.0.1:9092 \ - --supernode-address 127.0.0.1:9094 \ + --clientappio-api-address 127.0.0.1:9094 \ # In another terminal window, start another long-running SuperNode (at least 2 SuperNodes are required) $ flower-supernode \ --insecure \ --superlink 127.0.0.1:9092 \ - --supernode-address 127.0.0.1:9095 \ + --clientappio-api-address 127.0.0.1:9095 \ - Here's another example to start both SuperLink and SuperNodes with HTTPS. Use the @@ -255,14 +255,14 @@ Deployment # In a new terminal window, start a long-running SuperNode $ flower-supernode \ --superlink 127.0.0.1:9092 \ - --supernode-address 127.0.0.1:9094 \ + --clientappio-api-address 127.0.0.1:9094 \ --root-certificates \ # In another terminal window, start another long-running SuperNode (at least 2 SuperNodes are required) $ flower-supernode \ --superlink 127.0.0.1:9092 \ - --supernode-address 127.0.0.1:9095 \ + --clientappio-api-address 127.0.0.1:9095 \ --root-certificates \ From b3019e6cb9a90d7276b29a4e9463da9da297bf74 Mon Sep 17 00:00:00 2001 From: "Daniel J. Beutel" Date: Wed, 20 Nov 2024 15:03:08 +0100 Subject: [PATCH 24/26] Edit --- doc/source/how-to-upgrade-to-flower-1.13.rst | 84 ++++++++++---------- 1 file changed, 42 insertions(+), 42 deletions(-) diff --git a/doc/source/how-to-upgrade-to-flower-1.13.rst b/doc/source/how-to-upgrade-to-flower-1.13.rst index 767c2ba7cff5..cd42f55cf8db 100644 --- a/doc/source/how-to-upgrade-to-flower-1.13.rst +++ b/doc/source/how-to-upgrade-to-flower-1.13.rst @@ -266,8 +266,8 @@ Deployment --root-certificates \ -Simulation in CLI -~~~~~~~~~~~~~~~~~ +Simulation (CLI) +~~~~~~~~~~~~~~~~ Wrap your existing client and strategy with |clientapp_link|_ and |serverapp_link|_, respectively. There is no need to use |startsim_link|_ anymore. Here's an example: @@ -333,12 +333,13 @@ respectively. There is no need to use |startsim_link|_ anymore. Here's an exampl Depending on your Flower version, you can run your simulation as follows: -- For Flower versions 1.11 and onwards, run ``flwr run`` in CLI. -- For Flower versions between 1.8 to 1.10, run |flower_simulation_link|_ in CLI and - point to the ``server_app`` / ``client_app`` object in the code instead of executing - the Python script. In the code snippet below, there is an example (assuming the - ``server_app`` and ``client_app`` objects are in a ``sim.py`` module). -- For Flower versions before 1.8, run the Python script directly. +- For Flower 1.11 and later, run ``flwr run`` in the terminal. This is the recommended + way to start simulations, other ways are deprecated and no longer recommended. +- DEPRECATED For Flower versions between 1.8 and 1.10, run |flower_simulation_link|_ in + the terminal and point to the ``server_app`` / ``client_app`` object in the code + instead of executing the Python script. In the code snippet below, there is an example + (assuming the ``server_app`` and ``client_app`` objects are in a ``sim.py`` module). +- DEPRECATED For Flower versions before 1.8, run the Python script directly. .. code-block:: bash :emphasize-lines: 2 @@ -347,11 +348,11 @@ Depending on your Flower version, you can run your simulation as follows: $ flwr run - # Flower 1.8 - 1.10 (deprecated, no longer supported) - $ flower-simulation \ - --server-app=sim:server_app \ - --client-app=sim:client_app \ - --num-supernodes=10 + # # Flower 1.8 - 1.10 (deprecated, no longer supported) + # $ flower-simulation \ + # --server-app=sim:server_app \ + # --client-app=sim:client_app \ + # --num-supernodes=10 # Flower 1.7 (deprecated) @@ -359,13 +360,13 @@ Depending on your Flower version, you can run your simulation as follows: Depending on your Flower version, you can also define the default resources as follows: -- For Flower versions 1.11 and onwards, you can edit your ``pyproject.toml`` file and - then run ``flwr run`` in CLI as shown in the example below. -- For Flower versions between 1.8 to 1.10, you can adjust the resources for each - |clientapp_link|_ using the ``--backend-config`` command line argument instead of +- For Flower 1.11 and later, you can edit your ``pyproject.toml`` file and then run + ``flwr run`` in the terminal as shown in the example below. +- DEPRECATED For Flower versions between 1.8 and 1.10, you can adjust the resources for + each |clientapp_link|_ using the ``--backend-config`` command line argument instead of setting the ``client_resources`` argument in |startsim_link|_. -- For Flower versions before 1.8, you need to run |startsim_link|_ and pass a dictionary - of the required resources to the ``client_resources`` argument. +- DEPRECATED For Flower versions before 1.8, you need to run |startsim_link|_ and pass a + dictionary of the required resources to the ``client_resources`` argument. .. code-block:: bash :emphasize-lines: 2,8 @@ -379,12 +380,12 @@ Depending on your Flower version, you can also define the default resources as f $ flwr run - # Flower 1.8 - 1.10 (deprecated, no longer supported) - $ flower-simulation \ - --client-app=sim:client_app \ - --server-app=sim:server_app \ - --num-supernodes=10 \ - --backend-config='{"client_resources": {"num_cpus": 2, "num_gpus": 0.25}}' + # # Flower 1.8 - 1.10 (deprecated, no longer supported) + # $ flower-simulation \ + # --client-app=sim:client_app \ + # --server-app=sim:server_app \ + # --num-supernodes=10 \ + # --backend-config='{"client_resources": {"num_cpus": 2, "num_gpus": 0.25}}' .. code-block:: python @@ -394,18 +395,18 @@ Depending on your Flower version, you can also define the default resources as f num_clients=10, client_resources={"num_cpus": 2, "num_gpus": 0.25}, ... ) -Simulation in a Notebook -~~~~~~~~~~~~~~~~~~~~~~~~ +Simulation (Notebook) +~~~~~~~~~~~~~~~~~~~~~ To run your simulation from within a notebook, please consider the following examples depending on your Flower version: -- For Flower versions 1.11 and onwards, you need to run |runsim_link|_ in your notebook - instead of |startsim_link|_. -- For Flower versions between 1.8 to 1.10, you need to run |runsim_link|_ in your - notebook instead of |startsim_link|_ and configure the resources. -- For Flower versions before 1.8, you need to run |startsim_link|_ and pass a dictionary - of the required resources to the ``client_resources`` argument. +- For Flower 1.11 and later, you need to run |runsim_link|_ in your notebook instead of + |startsim_link|_. +- DEPRECATED For Flower versions between 1.8 and 1.10, you need to run |runsim_link|_ in + your notebook instead of |startsim_link|_ and configure the resources. +- DEPRECATED For Flower versions before 1.8, you need to run |startsim_link|_ and pass a + dictionary of the required resources to the ``client_resources`` argument. .. tip:: @@ -413,7 +414,7 @@ depending on your Flower version: |flower_how_to_run_simulations_link|_ guide. .. code-block:: python - :emphasize-lines: 2,6,10,14 + :emphasize-lines: 10,12,14-17 from flwr.client import ClientApp from flwr.common import Context @@ -422,9 +423,7 @@ depending on your Flower version: # Flower 1.10 and later (recommended) - def client_fn(context: Context): - return FlowerClient().to_client() - + # Omitted: client_fn and server_fn client_app = ClientApp(client_fn=client_fn) @@ -460,7 +459,8 @@ depending on your Flower version: # backend_config=backend_config, # ) - # Flower v1.7 + + # Flower v1.7 (deprecated) NUM_CLIENTS = 10 # Replace by any integer greater than zero backend_config = {"client_resources": {"num_cpus": 2, "num_gpus": 0.25}} start_simulation( @@ -474,12 +474,12 @@ depending on your Flower version: Further help ------------ -Some official `Flower code examples `_ are already +Most official `Flower code examples `_ are already updated to Flower 1.13 so they can serve as a reference for using the Flower 1.13 API. If there are further questions, `join the Flower Slack `_ -and use the channel ``#questions``. You can also `participate in Flower Discuss -`_ where you can find us answering questions, or share and -learn from others about migrating to Flower 1.13. +(and use the channel ``#questions``) or post them on `Flower Discuss +`_ where you can find the community posting and answering +questions. .. admonition:: Important From c732bf5c923b559aefae1f10591ecced76852041 Mon Sep 17 00:00:00 2001 From: Chong Shen Ng Date: Wed, 20 Nov 2024 14:16:51 +0000 Subject: [PATCH 25/26] Remove links to deprecated APIs --- doc/source/how-to-upgrade-to-flower-1.13.rst | 40 +++++--------------- 1 file changed, 10 insertions(+), 30 deletions(-) diff --git a/doc/source/how-to-upgrade-to-flower-1.13.rst b/doc/source/how-to-upgrade-to-flower-1.13.rst index cd42f55cf8db..05aa3256cb26 100644 --- a/doc/source/how-to-upgrade-to-flower-1.13.rst +++ b/doc/source/how-to-upgrade-to-flower-1.13.rst @@ -21,18 +21,8 @@ Let's dive in! .. |serverapp_link| replace:: ``ServerApp`` -.. |startclient_link| replace:: ``start_client()`` - -.. |startserver_link| replace:: ``start_server()`` - -.. |startsim_link| replace:: ``start_simulation()`` - .. |runsim_link| replace:: ``run_simulation()`` -.. |flower_clientapp_link| replace:: ``flower-client-app`` - -.. |flower_serverapp_link| replace:: ``flower-server-app`` - .. |flower_superlink_link| replace:: ``flower-superlink`` .. |flower_supernode_link| replace:: ``flower-supernode`` @@ -47,12 +37,8 @@ Let's dive in! .. _flower_architecture_link: explanation-flower-architecture.html -.. _flower_clientapp_link: ref-api-cli.html#flower-client-app - .. _flower_how_to_run_simulations_link: how-to-run-simulations.html -.. _flower_serverapp_link: ref-api-cli.html#flower-server-app - .. _flower_simulation_link: ref-api-cli.html#flower-simulation .. _flower_superlink_link: ref-api-cli.html#flower-superlink @@ -63,12 +49,6 @@ Let's dive in! .. _serverapp_link: ref-api/flwr.server.ServerApp.html -.. _startclient_link: ref-api/flwr.client.start_client.html - -.. _startserver_link: ref-api/flwr.server.start_server.html - -.. _startsim_link: ref-api/flwr.simulation.start_simulation.html - Install update -------------- @@ -134,7 +114,7 @@ project both in the traditional (now deprecated) way and in the new (recommended ~~~~~~~~~~~~~~~~~ - Wrap your existing client with |clientapp_link|_ instead of launching it via - |startclient_link|_. Here's an example: + ``start_client()``. Here's an example: .. code-block:: python :emphasize-lines: 6,10 @@ -170,7 +150,7 @@ project both in the traditional (now deprecated) way and in the new (recommended ~~~~~~~~~~~~~~~~~ - Wrap your existing strategy with |serverapp_link|_ instead of starting the server via - |startserver_link|_. Here's an example: + ``start_server()``. Here's an example: .. code-block:: python :emphasize-lines: 7,13 @@ -270,7 +250,7 @@ Simulation (CLI) ~~~~~~~~~~~~~~~~ Wrap your existing client and strategy with |clientapp_link|_ and |serverapp_link|_, -respectively. There is no need to use |startsim_link|_ anymore. Here's an example: +respectively. There is no need to use ``start_simulation()`` anymore. Here's an example: .. tip:: @@ -364,9 +344,9 @@ Depending on your Flower version, you can also define the default resources as f ``flwr run`` in the terminal as shown in the example below. - DEPRECATED For Flower versions between 1.8 and 1.10, you can adjust the resources for each |clientapp_link|_ using the ``--backend-config`` command line argument instead of - setting the ``client_resources`` argument in |startsim_link|_. -- DEPRECATED For Flower versions before 1.8, you need to run |startsim_link|_ and pass a - dictionary of the required resources to the ``client_resources`` argument. + setting the ``client_resources`` argument in ``start_simulation()``. +- DEPRECATED For Flower versions before 1.8, you need to run ``start_simulation()`` and + pass a dictionary of the required resources to the ``client_resources`` argument. .. code-block:: bash :emphasize-lines: 2,8 @@ -402,11 +382,11 @@ To run your simulation from within a notebook, please consider the following exa depending on your Flower version: - For Flower 1.11 and later, you need to run |runsim_link|_ in your notebook instead of - |startsim_link|_. + ``start_simulation()``. - DEPRECATED For Flower versions between 1.8 and 1.10, you need to run |runsim_link|_ in - your notebook instead of |startsim_link|_ and configure the resources. -- DEPRECATED For Flower versions before 1.8, you need to run |startsim_link|_ and pass a - dictionary of the required resources to the ``client_resources`` argument. + your notebook instead of ``start_simulation()`` and configure the resources. +- DEPRECATED For Flower versions before 1.8, you need to run ``start_simulation()`` and + pass a dictionary of the required resources to the ``client_resources`` argument. .. tip:: From ba487bfad12c90519c30024278ecb86b603348ef Mon Sep 17 00:00:00 2001 From: Chong Shen Ng Date: Wed, 20 Nov 2024 14:17:59 +0000 Subject: [PATCH 26/26] Remove links to deprecated APIs --- doc/source/how-to-upgrade-to-flower-1.13.rst | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/doc/source/how-to-upgrade-to-flower-1.13.rst b/doc/source/how-to-upgrade-to-flower-1.13.rst index 05aa3256cb26..41cb6efc8341 100644 --- a/doc/source/how-to-upgrade-to-flower-1.13.rst +++ b/doc/source/how-to-upgrade-to-flower-1.13.rst @@ -31,16 +31,12 @@ Let's dive in! .. |flower_how_to_run_simulations_link| replace:: How-to Run Simulations -.. |flower_simulation_link| replace:: ``flower-simulation`` - .. _clientapp_link: ref-api/flwr.client.ClientApp.html .. _flower_architecture_link: explanation-flower-architecture.html .. _flower_how_to_run_simulations_link: how-to-run-simulations.html -.. _flower_simulation_link: ref-api-cli.html#flower-simulation - .. _flower_superlink_link: ref-api-cli.html#flower-superlink .. _flower_supernode_link: ref-api-cli.html#flower-supernode @@ -315,9 +311,9 @@ Depending on your Flower version, you can run your simulation as follows: - For Flower 1.11 and later, run ``flwr run`` in the terminal. This is the recommended way to start simulations, other ways are deprecated and no longer recommended. -- DEPRECATED For Flower versions between 1.8 and 1.10, run |flower_simulation_link|_ in - the terminal and point to the ``server_app`` / ``client_app`` object in the code - instead of executing the Python script. In the code snippet below, there is an example +- DEPRECATED For Flower versions between 1.8 and 1.10, run ``flower-simulation`` in the + terminal and point to the ``server_app`` / ``client_app`` object in the code instead + of executing the Python script. In the code snippet below, there is an example (assuming the ``server_app`` and ``client_app`` objects are in a ``sim.py`` module). - DEPRECATED For Flower versions before 1.8, run the Python script directly.