The algorithm is based on paper "PDSS: A Privacy-Preserving Framework for Step-by-Step Distillation of Large Language Models", which introduce a novel framework for privacy preserving federated distillation. We integrate its code into the FATE-LLM framework.
If you publish work that uses PDSS, please cite PDSS as follows:
@article{fan2024pdss,
title={PDSS: A Privacy-Preserving Framework for Step-by-Step Distillation of Large Language Models},
author={Fan, Tao and Kang, Yan and Chen, Weijing and Gu, Hanlin and Song, Yuanfeng and Fan, Lixin and Chen, Kai and Yang, Qiang},
journal={arXiv preprint arXiv:2406.12403},
year={2024}
}