diff --git a/baselines/FedMeta/README.md b/baselines/FedMeta/README.md index b279fbe4a5fd..ba0630c74e15 100644 --- a/baselines/FedMeta/README.md +++ b/baselines/FedMeta/README.md @@ -33,17 +33,30 @@ dataset: [FEMNIST, SHAKESPEARE] # list of datasets you include in your baseline * A two-layer CNN network as used in the FedMeta paper (see `models/CNN_Network`). This is the model used by default. * A StackedLSTM model used in the FedMeta paper for Shakespeare (see `models/StackedLSTM`). -**You can see more detail model at Apendix.A in paper** +**You can see more detail at Apendix.A in paper** -****Dataset:**** : This baseline includes the FEMNIST dataset and SHAKESPEARE. Now you should include a breakdown of the details about each of them. Please include information about: how the dataset is partitioned (e.g. LDA with alpha 0.1 as default and all clients have the same number of training examples; or each client gets assigned a different number of samples following a power-law distribution with each client only instances of 2 classes)? if your dataset is naturally partitioned just state “naturally partitioned”; how many partitions there are (i.e. how many clients)? Please include this an all information relevant about the dataset and its partitioning into a table. +****Dataset:**** : This baseline includes the FEMNIST dataset and SHAKESPEARE. For data partitioning and sampling per client, we use the Leaf GitHub([LEAF: A Benchmark for Federated Settings](https://github.com/TalwalkarLab/leaf)). The data and client specifications used in this experiment are listed in the table below (Table 1 in the paper). -| Dataset | #Clients | #Samples | #Classes | partition settings | -|:-----------:|:--------:| :---: |:--------:|:---------------------------------------------:| -| FEMNIST | 1,068 | 235,683 | 62 | Support set : 0.2, Query set : 0.8 | -| SHAKESPEARE | 110 | 625,127 | 80 | Support set : 0.2, Query set : 0.8 | +| Dataset | #Clients | #Samples | #Classes | #Partition Clients | #Partition Dataset | +|:-----------:|:--------:| :---: |:--------:|:------------------------------------------------------------:|-----------------------------------| +| FEMNIST | 1,068 | 235,683 | 62 | Train Clients : 0.8, Valid Clients : 0.1, Test Clients : 0.1 | Support set : 0.8, Query set : 0.2 | +| SHAKESPEARE | 110 | 625,127 | 80 | Train Clients : 0.8, Valid Clients : 0.1, Test Clients : 0.1 | Support set : 0.8, Query set : 0.2| + +**The original specifications of the Leaf dataset can be found in the Leaf paper(_"LEAF: A Benchmark for Federated Settings"_).** ****Training Hyperparameters:**** :warning: *_Include a table with all the main hyperparameters in your baseline. Please show them with their default value._* +| Algorithm | Dataset | clients per round | number of rounds | batch size | optimizer | Learning Rate(α, β) | client resources | +|:-----------------:|:--------------:|:-------------------:|:------------------:|:-----------:|:---------:|:-------------------:|--------------------------------------| +| FedAvg | FEMNST | 4(Defalut), 40, 50 | 2000 | 10 | Adam | 0.0001 | {'num_cpus': 4.0, 'num_gpus': 0.25 } | +| FedAVg | SHAKESPEARE | 4(Defalut), 40, 50 | 400 | 10 | Adam | 0.001 | {'num_cpus': 4.0, 'num_gpus': 0.25 } | +| FedAvg(Meta) | FEMNST | 4(Defalut), 40, 50 | 2000 | 10 | Adam | 0.0001 | {'num_cpus': 4.0, 'num_gpus': 0.25 } | +| FedAvg(Meta) | SHAKESPEARE | 4(Defalut), 40, 50 | 400 | 10 | Adam | 0.001 | {'num_cpus': 4.0, 'num_gpus': 0.25 } | +| FedMeta(MAML) | FEMNST | 4(Defalut), 40, 50 | 2000 | 10 | Adam | (0.001,0.0001) | {'num_cpus': 4.0, 'num_gpus': 0.25 } | +| FedMeta(MAML) | SHAKESPEARE | 4(Defalut), 40, 50 | 400 | 10 | Adam | (0.1,0.01) | {'num_cpus': 4.0, 'num_gpus': 1.0 } | +| FedMeta(Meta-SGD | FEMNST | 4(Defalut), 40, 50 | 2000 | 10 | Adam | (0.001,0.0001) | {'num_cpus': 4.0, 'num_gpus': 0.25 } | +| FedMeta(Meta-SGD | SHAKESPEARE | 4(Defalut), 40, 50 | 400 | 10 | Adam | (0.1,0.01) | {'num_cpus': 4.0, 'num_gpus': 1.0 } | + ## Environment Setup