Merge branch 'main' into add-datasets-ci-tests

.github/workflows/release-nightly.yml

@@ -0,0 +1,22 @@
+name: Release nightly
+
+on:
+  schedule:
+    - cron: "0 23 * * *"
+
+env:
+  FLWR_TELEMETRY_ENABLED: 0
+
+jobs:
+  release_nightly:
+    runs-on: ubuntu-22.04
+    name: Nightly
+    steps:
+      - uses: actions/checkout@v4
+      - name: Bootstrap
+        uses: ./.github/actions/bootstrap
+      - name: Release nightly
+        env:
+          PYPI_TOKEN: ${{ secrets.PYPI_TOKEN }}
+        run: |
+          ./dev/publish-nightly.sh

baselines/fedprox/README.md

@@ -59,6 +59,13 @@ The following table shows the main hyperparameters for this baseline with their
 To construct the Python environment, simply run:
 
 ```bash
+# Set directory to use python 3.10 (install with `pyenv install <version>` if you don't have it)
+pyenv local 3.10.12
+
+# Tell poetry to use python3.10
+poetry env use 3.10.12
+
+# Install
 poetry install
 ```
 
@@ -97,6 +104,6 @@ python -m fedprox.main --multirun mu=0.0,2.0 stragglers_fraction=0.0,0.5,0.9 '+r
 python -m fedprox.main --config-name fedavg --multirun stragglers_fraction=0.0,0.5,0.9 '+repeat_num=range(5)'
 ```
 
-The above commands would generate results that you can plot and would look like:
+The above commands would generate results that you can plot and would look like the plot shown below. This plot was generated using the jupyter notebook in the `docs/` directory of this baseline after running the `--multirun` commands above.
 
-![](docs/FedProx_mnist.png)
+![](_static/FedProx_mnist.png)

baselines/fedprox/fedprox/models.py

@@ -55,8 +55,9 @@ class LogisticRegression(nn.Module):
 
     As described in the Li et al., 2020 paper :
 
-    [Federated Optimization in Heterogeneous Networks]
-    (https://arxiv.org/pdf/1812.06127.pdf)
+    [Federated Optimization in Heterogeneous Networks] (
+
+    https://arxiv.org/pdf/1812.06127.pdf)
     """
 
     def __init__(self, num_classes: int) -> None:
@@ -153,7 +154,7 @@ def _train_one_epoch(  # pylint: disable=too-many-arguments
         optimizer.zero_grad()
         proximal_term = 0.0
         for local_weights, global_weights in zip(net.parameters(), global_params):
-            proximal_term += (local_weights - global_weights).norm(2)
+            proximal_term += torch.square((local_weights - global_weights).norm(2))
         loss = criterion(net(images), labels) + (proximal_mu / 2) * proximal_term
         loss.backward()
         optimizer.step()

baselines/fedprox/pyproject.toml

@@ -41,6 +41,8 @@ python = ">=3.10.0, <3.11.0"
 flwr = { extras = ["simulation"], version = "1.5.0" }
 hydra-core = "1.3.2"
 matplotlib = "3.7.1"
+jupyter = "^1.0.0"
+pandas = "^2.0.3"
 torch = { url = "https://download.pytorch.org/whl/cu117/torch-2.0.1%2Bcu117-cp310-cp310-linux_x86_64.whl"}
 torchvision = { url = "https://download.pytorch.org/whl/cu117/torchvision-0.15.2%2Bcu117-cp310-cp310-linux_x86_64.whl"}
 

datasets/doc/source/tutorial-quickstart.rst

@@ -0,0 +1,72 @@
+Quickstart
+==========
+
+Run Flower Datasets as fast as possible by learning only the essentials.
+
+Install Federated Datasets
+--------------------------
+Run on the command line::
+
+  python -m pip install flwr-datasets[vision]
+
+Install the ML framework
+------------------------
+TensorFlow::
+
+  pip install tensorflow
+
+PyTorch::
+
+  pip install torch torchvision
+
+Choose the dataset
+------------------
+Choose the dataset by going to Hugging Face `Datasets Hub <https://huggingface.co/datasets>`_ and searching for your
+dataset by name. Note that the name is case sensitive, so make sure to pass the correct name as the `dataset` parameter
+to `FederatedDataset`.
+
+Partition the dataset
+---------------------
+::
+
+  from flwr_datasets import FederatedDataset
+
+  fds = FederatedDataset(dataset="cifar10", partitioners={"train": 10})
+  partition = fds.load_partition(0, "train")
+  centralized_dataset = fds.load_full("test")
+
+Now you're ready to go. You have ten partitions created from the train split of the MNIST dataset and the test split
+for the centralized evaluation. We will convert the type of the dataset from Hugging Face's Dataset type to the one
+supported by your framework.
+
+Conversion
+----------
+For more detailed instructions, go to :doc:`how-to`.
+
+PyTorch DataLoader
+^^^^^^^^^^^^^^^^^^
+Transform the Dataset directly into the DataLoader::
+
+  from torch.utils.data import DataLoader
+  from torchvision.transforms import ToTensor
+
+  transforms = ToTensor()
+  partition_torch = partition.map(
+        lambda img: {"img": transforms(img)}, input_columns="img"
+    ).with_format("torch")
+  dataloader = DataLoader(partition_torch, batch_size=64)
+
+NumPy
+^^^^^
+NumPy can be used as input to the TensorFlow model and is very straightforward::
+
+   partition_np = partition.with_format("numpy")
+   X_train, y_train = partition_np["img"], partition_np["label"]
+
+TensorFlow Dataset
+^^^^^^^^^^^^^^^^^^
+Transformation to TensorFlow Dataset is a one-liner::
+
+  tf_dataset = partition.to_tf_dataset(columns="img", label_cols="label", batch_size=64,
+                                     shuffle=True)
+

doc/source/ref-changelog.md

@@ -14,7 +14,7 @@
 
 - **Update Flower Baselines**
 
-  - FedProx ([#2286](https://github.com/adap/flower/pull/2286))
+  - FedProx ([#2210](https://github.com/adap/flower/pull/2210), [#2286](https://github.com/adap/flower/pull/2286))
 
 - **General updates to baselines** ([#2301](https://github.com/adap/flower/pull/2301).[#2305](https://github.com/adap/flower/pull/2305), [#2307](https://github.com/adap/flower/pull/2307), [#2327](https://github.com/adap/flower/pull/2327))
 

doc/source/tutorial-series-what-is-federated-learning.ipynb

@@ -15,7 +15,7 @@
     "\n",
     "> [Star Flower on GitHub](https://github.com/adap/flower) ⭐️ and join the open-source Flower community on Slack to connect, ask questions, and get help: [Join Slack](https://flower.dev/join-slack) 🌼 We'd love to hear from you in the `#introductions` channel! And if anything is unclear, head over to the `#questions` channel.\n",
     "\n",
-    "Let's get stated!"
+    "Let's get started!"
    ]
   },
   {

pyproject.toml

@@ -70,8 +70,8 @@ ray = { version = "==2.6.3", extras = ["default"], optional = true }
 pydantic = { version = "<2.0.0", optional = true }
 # Optional dependencies (REST transport layer)
 requests = { version = "^2.31.0", optional = true }
-starlette = { version = "^0.29.0", optional = true }
-uvicorn = { version = "^0.22.0", extras = ["standard"], optional = true }
+starlette = { version = "^0.31.0", optional = true }
+uvicorn = { version = "^0.23.0", extras = ["standard"], optional = true }
 
 [tool.poetry.extras]
 simulation = ["ray", "pydantic"]