Table of Contents
This framework can be used to simulate and analyse a federated learning setting in which some clients are compromised by an adversary. The adversary attempts to compromise the integrity of the shared global model by sending malicious updates to the server.
The framework was used to perform the analysis of federated learning robustness under a norm bound defense as part of RoFL: Attestable Robustness for Secure Federated Learning. A research implementation of the secure federated learning with constraints framework can be found here.
In federated learning, adversaries can perform backdoor attacks to poison the global model. This framework implements existing attack strategies such as model replacement, on a wide variety of tasks and backdoor attack targets proposed in previous work, such as attacks on prototypical targets or edge cases.
The framework provides several tools to analyse client updates, measure backdoor performance and deploy defenses to gain insight on model robustness in federated learning.
We now describe how to set up this framework.
The dependencies can be automatically installed through pipenv.
The high-level requirements are as follows.
- Python 3 (tested on version 3.7)
- TensorFlow (version 2.0)
Before starting, ensure that you have pipenv installed:
pip install pipenv- Clone the repo
git clone https://github.com/pps-lab/fl-analysis.git- Install the Python packages
pipenv installThe configuration of the framework is specified in a config file in YAML format. A minimal example of a config is shown below.
environment:
num_clients: 3383
num_selected_clients: 30
num_malicious_clients: 0
experiment_name: "Sample run without attackers"
server:
num_rounds: 80
num_test_batches: 5
aggregator:
name: FedAvg
global_learning_rate: -1
client:
clip:
type: l2
value: 10
model_name: resnet18
benign_training:
num_epochs: 2
batch_size: 24
optimizer: Adam
learning_rate: 0.001
dataset:
dataset: femnist
data_distribution: nonIIDThe full specification of the supported config options can be found here
Some example config files can be find in train_configs.
With a config file config.yml ready, the framework can be started by invoking:
python -m src.main -c config.yml
Some pre-trained models are available in the models for experiments and can be included in training using the environment.load_model config key.
lenet5_emnist_088.h5LeNet5 for federated-MNIST at 0.88 accuracy.lenet5_emnist_097.h5LeNet5 for federated-MNIST at 0.97 accuracy.lenet5_emnist_098.h5LeNet5 for federated-MNIST at 0.98 accuracy.resnet18.h5ResNet18 for CIFAR-10 at 0.88 accuracy.resnet18_080.h5ResNet18 for CIFAR-10 at 0.80 accuracy.resnet18_082.h5ResNet18 for CIFAR-10 at 0.82 accuracy.resnet156_082.h5ResNet56 for CIFAR-10 at 0.86 accuracy.
Basic training progress is sent to standard output.
More elaborate information is stored in an output folder.
The directory location can be specified through the XXX option.
By default, its ... .
The framework stores progress in tfevents, which can be viewed using Tensorboard, e.g.,
tensorboard --logdir ./experiments/{experiment_name}This project's code is distributed under the MIT License. See LICENSE for more information.