Awesome Secure Collaborative Learning Papers

A curated list of robust and privacy-preserving collaborative learning publications, organized by the order System Overview, Integrity Attacks and Defenses, Privacy Threats and Defenses

Table of Content

• Awesome Secure Collaborative Learning Papers
  • Table of Content
  • System Overview
    • Centralized
    • Decentralized
    • Federated Learning
    • Application
    • Survey
  • Integrity Attacks
    • Data Poison
    • Model Poison
  • Integrity Defenses
    • Data Based Inspection
    • Model Based Inspection
  • Privacy Threats
    • Membership Inference Attacks
    • Property Inference Attacks
    • Sample Inference Attacks
    • Membership & Property
  • Privacy Defenses
    • Differentially Private Defences
    • Cryptographic Privacy Defenses
      • Homomorphic Encryption(HE)
      • Secure Muti-Party Computation(SMC)
      • DP & SMC
      • DP & Trusted Hardware
      • Function Encryption(FE)
    • Practical Privacy Defenses
  • Hybrid Defenses and Beyond
    • Hybrid Defenses
    • Collaborative Adversarial Training

System Overview

Centralized

• Exploiting GPUs for Efficient Gradient Boosting Decision Tree Training , TPDS 2020
• SPARKNET: TRAINING DEEP NETWORKS IN SPARK , ICLR 2016
• Scaling Distributed Machine Learning with the Parameter Server , USENIX 2014
• More Effective Distributed ML via a Stale Synchronous Parallel Parameter Server , NeurIPS 2013
• Large Scale Distributed Deep Networks , NeurIPS 2012

Decentralized

• Decentralized Learning With Lazy and Approximate Dual Gradients, IEEE Trans. Signal Process. 2021
• Consensus Control for Decentralized Deep Learning, ICML 2021
• Cross-Gradient Aggregation for Decentralized Learning from Non-IID data, ICML 2021
• Byzantine-Resilient Decentralized Policy Evaluation With Linear Function Approximation, IEEE Trans. Signal Process. 2021
• Stability and Generalization of Decentralized Stochastic Gradient Descent, AAAI 2021
• Optimal and Practical Algorithms for Smooth and Strongly Convex Decentralized Optimization, NeurIPS 2020
• A Decentralized Parallel Algorithm for Training Generative Adversarial Nets , NeurIPS 2020
• Communication Compression for Decentralized Training, NeurIPS 2018
• an Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent, NeurIPS 2017
• Fully Decentralized Policies for Multi-Agent Systems: An Information Theoretic Approach, NeurIPS 2017

Federated Learning

• Curse or Redemption? How Data Heterogeneity Affects the Robustness of Federated Learning , AAAI 2021
• On the Convergence of Communication-Efficient Local SGD for Federated Learning , AAAI 2021
• Federated Bayesian Optimization via Thompson Sampling , NeurIPS 2020
• Performance Optimization of Federated Person Re-identification via Benchmark Analysis , MM 2020
• Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization , NeurIPS 2020
• SCAFFOLD: Stochastic Controlled Averaging for Federated Learning , ICML 2020
• Practical Federated Gradient Boosting Decision Trees , AAAI 2020
• Personalized Federated Learning with Theoretical Guarantees: A Model-Agnostic Meta-Learning Approach , NeurIPS 2020
• Towards Federated Learning at Scale: System Design , SysML 2019
• Adaptive Kernel Value Caching for SVM Training , IEEE Trans Neural Netw Learn Syst 2019
• A Performance Evaluation of Federated Learning Algorithms , DIDL 2018
• Federated Multi-Task Learning , NeurIPS 2017

Application

• Model-Contrastive Federated Learning , CVPR 2021
• Federated Meta-Learning for Fraudulent Credit Card Detection , IJCAI 2020
• Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge , NeurIPS 2020
• Privacy Regulation Aware Process Mapping in Geo-Distributed Cloud Data Centers , TPDS 2019
• Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge , ICC 2019
• Federated Learning for Keyword Spotting , ICASSP 2019
• Imagenet classification with deep convolutional neural networks , NeurIPS 2012

Survey

• Advances and open problems in federated learning , Found. Trends Mach. Learn. 2021
• A Survey on Federated Learning: The Journey From Centralized to Distributed On-Site Learning and Beyond , IEEE Internet Things J. 2021
• A survey on federated learning systems: vision, hyper and reality for data privacy and protection ,IEEE Trans Knowl Data Eng 2021
• A survey on security and privacy of federated learning, Future Gener. Comput. Syst. 2021
• A Comprehensive Survey of Privacy-preserving Federated Learning: A Taxonomy, Review, and Future Directions, ACM Comput. Surv. 2021
• Privacy and Robustness in Federated Learning: Attacks and Defenses, arXiv 2020
• Threats to Federated Learning: A Survey, arXiv 2020

Integrity Attacks

Data Poison

• Better Trigger Inversion Optimization in Backdoor Scanning , CVPR 2022
• DEFEAT: Deep Hidden Feature Backdoor Attacks by Imperceptible Perturbation and Latent Representation Constraints , CVPR 2022
• Invisible backdoor attack with sample-specific triggers , CVPR 2021
• LIRA: Learnable, Imperceptible and Robust Backdoor Attacks , CVPR 2021
• PoisonGAN: Generative Poisoning Attacks Against Federated Learning in Edge Computing Systems , IEEE Internet Things J. 2021
• Poison Attacks on Federated Learning Based IoT Intrusion Detection System , DISS 2020
• MetaPoison: Practical General-purpose Clean-label Data Poisoning , NeurIPS 2020
• Clean-Label Backdoor Attacks on Video Recognition Models , CVPR 2020
• Data Poisoning Attacks on Federated Machine Learning , arXiv 2020
• Attack of the Tails: Yes, You Really Can Backdoor Federated Learning , arXiv 2020
• Transferable Clean-Label Poisoning Attacks on Deep Neural Nets , ICML 2019
• Clean-Label Backdoor Attacks , ICLR 2019
• DBA: DISTRIBUTED BACKDOOR ATTACKS AGAINST FEDERATED LEARNING , ICLR 2019
• Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networks , NeurIPS 2018
• Backdoor Attacks against Learning Systems , CNS 2017
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain , arXiv 2017
• Data Poisoning Attacks on Factorization-Based Collaborative Filtering , NeurIPS 2016
• Data Poisoning Attacks against Autoregressive Models , AAAI 2016

Model Poison

• Back to the Drawing Board: A Critical Evaluation of Poisoning Attacks on Production Federated Learning, Oakland 2022
• Local Model Poisoning Attacks to Byzantine-Robust Federated Learning , USENIX 2020
• How To Backdoor Federated Learning , ICML 2020
• Can You Really Backdoor Federated Learning? , arXiv 2019
• Analyzing Federated Learning through an Adversarial Lens , ICML 2019

Integrity Defenses

Data Based Inspection

• Complex Backdoor Detection by Symmetric Feature Differencing ,CVPR 2022
• FL-WBC: Enhancing Robustness against Model Poisoning Attacks in Federated Learning from a Client Perspective , NeurIPS 2021
• Scalable Backdoor Detection in Neural Networks , arXiv 2020 .
• Shielding Collaborative Learning: Mitigating Poisoning Attacks through Client-Side Detection , TDSC 2020 .
• SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systems , SPW 2020
• Systematic Evaluation of Backdoor Data Poisoning Attacks on Image Classifiers , CVPR 2020
• Ensemble distillation for robust model fusion in federated learning , NeurIPS 2020
• ABS: Scanning neural networks for back-doors by artificial brain stimulation , SIGSAC
• STRIP: A Defence Against Trojan Attacks on Deep Neural Networks , ACSAC 2019
• Poison as a Cure: Detecting & Neutralizing Variable-Sized Backdoor Attacks in Deep Neural Networks , arXiv 2019
• Cronus: Robust and heterogeneous collaborative learning with black-box knowledge transfer , arXiv 2019
• Spectral Signatures in Backdoor Attacks , NeurIPS 2018
• Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering , arXiv 2018

Model Based Inspection

• Deep learning backdoors , Security and Artificial Intelligence 2022
• DeepSight: Mitigating Backdoor Attacks in Federated Learning Through Deep Model Inspection , NDSS 2022
• Baffle: backdoor detection via feedback-based federated learning , ICDCS 2021
• Defending against backdoors in federated learning with robust learning rate , AAAI 2021
• CRFL: Certifiably Robust Federated Learning against Backdoor Attacks , ICML 2021
• Ditto: Fair and robust federated learning through personalization , ICML 2021
• Backdoor attacks and defenses in feature-partitioned collaborative learning , arXiv 2020
• ABS: Scanning Neural Networks for Back-doors by Artificial Brain Stimulation , CCS 2019
• NIC: Detecting Adversarial Samples with Neural Network Invariant Checking , NDSS 2019
• Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks , Oakland 2019
• NeuronInspect: Detecting Backdoors in Neural Networks via Output Explanations , arXiv 2019 .
• DeepInspect: A Black-box Trojan Detection and Mitigation Framework for Deep Neural Networks , IJCAI 2019

Privacy Threats

Membership Inference Attacks

• Beyond Class-Level Privacy Leakage: Breaking Record-Level Privacy in Federated Learning , IEEE Internet Things J. 2021
• Analyzing user-level privacy attack against federated learning , J-SAC 2020
• GAN Enhanced Membership Inference: A Passive LocalAttack in Federated Learning , ICC 2020
• Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning , Oakland 2019
• Membership Inference Attacks Against Machine Learning Models , Oakland 2017

Property Inference Attacks

• Property Inference from Poisoning ,Oakland 2022
• Leakage of Dataset Properties in Multi-Party Machine Learning , USENIX 2021
• Beyond inferring class representatives: User-level privacy leakage from federated learning , ICCC 2019

Sample Inference Attacks

• Robbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified Models, ICLR 2022
• Exploring the Security Boundary of Data Reconstruction via Neuron Exclusivity Analysis ,USENIX 2022
• Bayesian Framework for Gradient Leakage, ICLR 2022
• Auditing Privacy Defenses in Federated Learning via Generative Gradient Leakage, CVPR 2022
• GradViT: Gradient Inversion of Vision Transformers, CVPR 2022
• Understanding Training-Data Leakage from Gradients in Neural Networks for Image Classification ,NeurIPS 2021 Workshop
• R-gap: Recursive gradient attack on privacy ,ICLR 2021
• Gradient Inversion with Generative Image Prior, NeurIPS 2021
• See through Gradients: Image Batch Recovery via GradInversion, CVPR 2021
• Revealing and Protecting Labels in Distributed Training , NeurIPS 2021
• CAFE: Catastrophic Data Leakage in Vertical Federated Learning , NeurIPS 2021
• Label Inference Attacks Against Vertical Federated Learning , USENIX 2021
• Knowledge-Enriched Distributional Model Inversion Attacks , CVPR 2021
• Gradient Disaggregation: Breaking Privacy in Federated Learning by Reconstructing the User Participant Matrix , ICML 2021
• Rethinking privacy preserving deep learning: How to evaluate and thwart privacy attacks ,arXiv 2020
• Inverting Gradients–How easy is it to break privacy in federated learning , arXiv 2020
• iDLG: Improved Deep Leakage from Gradients , arXiv 2020
• Deep Leakage from Gradients , NeurIPS 2019

Membership & Property

• Exploiting Unintended Feature Leakage in Collaborative Learning , Oakland 2019

Privacy Defenses

Differentially Private Defences

• Differentially private byzantine-robust federated learning , IEEE Trans. Parallel Distrib. Syst. 2022
• Topology-aware differential privacy for decentralized image classification , IEEE T CIRC SYST VID 2021
• Evaluating gradient inversion attacks and defenses in federated learning , NeurIPS 2021
• Accurate Differentially Private Deep Learning on the Edge , IEEE Trans Parallel Distrib Syst. 2021
• User-Level Privacy-Preserving Federated Learning: Analysis and Performance Optimization , IEEE Trans Mob Comput. 2021
• Gradient-leakage resilient federated learning , ICDCS 2021
• Pain-FL: Personalized Privacy-Preserving Incentive for Federated Learning , IEEE J. Sel. Areas Commun. 2021
• Romoa: Robust Model Aggregation for the Resistance of Federated Learning to Model Poisoning Attacks , ESORICS 2021
• Privacy Threat and Defense for Federated Learning with Non-iid Data in AIoT , IEEE Trans Industr Inform. 2021
• Differentially Private and Communication Efficient Collaborative Learning, AAAI 2021
• FLAME: Differentially Private Federated Learning in the Shuffle Model , AAAI 2021
• Local Differential Privacy-Based Federated Learning for Internet of Things , IEEE Internet Things J. 2021
• Federated Learning with Local Differential Privacy: Trade-Offs Between Privacy, Utility, and Communication , ICASSP 2021
• LDP-FL: Practical Private Aggregation in Federated Learning with Local Differential Privacy , IJCAI 2021
• Federated Learning with Sparsification-Amplified Privacy and Adaptive Optimization , IJCAI 2021
• Attacks to Federated Learning: Responsive Web User Interface to Recover Training Data from User Gradients , ASIACCS 2020
• Federated Learning with Differential Privacy: Algorithms and Performance Analysis , TIFS 2020
• Protection against reconstruction and its applications in private federated learning , arXiv 2019
• Differentially private model publishing for deep learning , Oakland 2019
• Evaluating Differentially Private Machine Learning in Practice , USENIX 2019
• Differentially private distributed online learning , TKDE 2018
• Improving the Privacy and Accuracy of ADMM-Based Distributed Algorithms , ICML 2018
• DP-EM: Differentially Private Expectation Maximization , ICML 2017
• Differential privacy preservation for deep auto-encoders: An application of human behavior prediction , AAAI 2016
• Deep learning with differential privacy , CCS 2016
• Privacy-preserving deep learning , CCS 2015
• Differentially private empirical risk minimization , JMLR 2011

Cryptographic Privacy Defenses

Homomorphic Encryption(HE)

• Efficient dropout-resilient aggregation for privacy-preserving machine learning ,IEEE Trans. Inf. Forensics Secur 2022
• Secure neuroimaging analysis using federated learning with homomorphic encryption , SIPAIM 2021
• GALA: Greedy ComputAtion for Linear Algebra in Privacy-Preserved Neural Networks , NDSS 2021
• Batchcrypt: Efficient homomorphic encryption for cross-silo federated learning , USENIX 2020
• Privacy-Preserving Federated Deep Learning with Irregular Users , IEEE Trans Dependable Secure Comput 2020
• Secure Logistic Regression Based on Homomorphic Encryption: Design and Evaluation, JMIR 2018
• Privacy-Preserving Deep Learning via Additively Homomorphic Encryption, IEEE Trans.Inform.Forensic Secur. 2018
• Oblivious Neural Network Predictions via MiniONN Transformations , CCS 2017
• SecureML: A System for Scalable Privacy-Preserving Machine Learning, Oakland 2017
• Scalable and Secure Logistic Regression via Homomorphic Encryption, CODASPY 2016

Secure Muti-Party Computation(SMC)

• Privacy-Preserving Federated Learning Framework Based on Chained Secure Multiparty Computing , IEEE Internet Things J. 2021
• Keep Your Data Locally: Federated-Learning-Based Data Privacy Preservation in Edge Computing , IEEE Network 2021
• Toward secure and privacy-preserving distributed deep learning in fog-cloud computing , IEEE Internet Things J. 2020
• Secure single-server aggregation with (poly) logarithmic overhead , IEEE Trans. Inf. Forensics Secur 2020
• HybridAlpha: An Efficient Approach for Privacy-Preserving Federated Learning , AISec 2019
• Chameleon: A Hybrid Secure Computation Framework for Machine Learning Applications , CCS 2018
• Practical Secure Aggregation for Privacy-Preserving Machine Learning , CCS 2017

DP & SMC

• A Hybrid Approach to Privacy-Preserving Federated Learning , AISec 2019
• Distributed learning without distress: Privacy-preserving empirical risk minimization , NeurIPS 2018

DP & Trusted Hardware

• Efficient deep learning on multi-source private data , arXiv 2018

Function Encryption(FE)

• Privacy-Preserving Federated Learning via Functional Encryption, Revisited , TIFS 2023

Practical Privacy Defenses

• More than Enough is Too Much: Adaptive Defenses against Gradient Leakage in Production Federated Learning , INFOCOM 2023
• PRECODE-A Generic Model Extension to Prevent Deep Gradient Leakage , CVPR 2022
• Soteria: Provable defense against privacy leakage in federated learning from representation perspective ,CVPR 2021
• Provable Defense against Privacy Leakage in Federated Learning from Representation Perspective, CVPR 2021
• Privacy-preserving collaborative learning with automatic transformation search , CVPR 2021
• PrivateDL: Privacy-preserving collaborative deep learning against leakage from gradient sharing , International Journal of Intelligent Systems 2020
• Instahide: Instance-hiding schemes for private distributed learning ,ICML 2020
• Mixup: Beyond Empirical Risk Minimization ,ICLR 2018

Hybrid Defenses and Beyond

Hybrid Defenses

• Privacy-preserving Byzantine-robust federated learning , Comput. Stand 2022
• Privacy-preserving blockchain-based federated learning for traffic flow prediction , Future Gener Comput Syst 2021
• Privacy-Enhanced Federated Learning Against Poisoning Adversaries , IEEE Trans. Inf. Forensics Secur 2021
• DP-SIGNSGD: When Efficiency Meets Privacy and Robustness , ICASSP 2021
• FLOD: Oblivious Defender for Private Byzantine-Robust Federated Learning with Dishonest-Majority , ESORICS 2021
• Secure and Privacy-Preserving Federated Learning via Co-Utility , IEEE Internet Things J. 2021
• Toward robustness and privacy in federated learning: Experimenting with local and central differential privacy , arXiv 2020
• Robust aggregation for adaptive privacy preserving federated learning in healthcare , arXiv 2020

Collaborative Adversarial Training

• Recent Advances in Adversarial Training for Adversarial Robustness , arXiv 2021
• Federated Robustness Propagation: Sharing Adversarial Robustness in Federated Learning , arXiv 2021
• Adversarial training in communication constrained federated learning , arXiv 2021
• Adversarially Robust Federated Learning for Neural Networks , 2020
• Fast is better than free: Revisiting adversarial training , arXiv 2020
• Adversarial training for free , NeurIPS 2019