diff --git a/README.md b/README.md index 4a502a9..56c4996 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,5 @@ # Offsite-Tuning: Transfer Learning without Full Model [[paper]()] - +
## Abstract @@ -38,7 +38,7 @@ In this repository, you will find all the necessary components to reproduce the ## Results - Comparing existing fine-tuning approaches (top and middle) and Offsite-Tuning (bottom). (a) Traditionally, users send labeled data to model owners for fine-tuning, raising privacy concerns and incurring high computational costs. (b) Model owner sending the full model to the data owner is not practical, which threatens the ownership of the proprietary model, and it's not affordable for users to fine-tune the huge foundation model due to resource constraints. (c) Offsite-tuning offers a privacy-preserving and efficient alternative to traditional fine-tuning methods that require access to full model weights. - +
- On 1-billion scale language models, Offsite-tuning (OT Plug-in) improves zero-shot (ZS) performance across all tasks, with only slight decreases compared to full fine-tuning (FT). Also, a consistent performance gap is observed between the emulator fine-tuning and plug-in, indicating offsite-tuning effectively preserves the privacy of the original proprietary model (users can not use the emulator to achieve the same performance). ![lm_results](figures/lm_results.png) @@ -48,7 +48,7 @@ In this repository, you will find all the necessary components to reproduce the - Offsite-Tuning significantly increase the fine-tuning throughput and reduce the memory footprint compared to the existing fine-tuning methods. - +
## Citation diff --git a/figures/efficiency.png b/figures/efficiency.png index 9926de0..5170066 100644 Binary files a/figures/efficiency.png and b/figures/efficiency.png differ