Update README.md

README.md

@@ -1,17 +1,9 @@
-# offsite-tuning
+# Offsite-Tuning
 The official implementation of Offsite-Tuning.
 
 ## Abstract
 
-Foundation models have become increasingly popular due to their general performance on downstream tasks.
-Although some foundation models can make zero-shot predictions, transfer learning (e.g. fine-tuning) is still important for adaption to specific downstream tasks.
-However, many foundation models are proprietary, so users must share their data with model owners to fine-tune the models, which is costly and raise privacy concerns. Moreover, fine-tuning foundation models is computation-intensive and impractical for most downstream users.
-In this paper, we propose Offsite-Tuning, a privacy-preserving and efficient transfer learning framework that can adapt foundation models to downstream data without full model access.
-In offsite-tuning, the model owner sends a light-weight adapter and a lossy compressed emulator to the data owner, who then fine-tunes the adapter on the downstream data with the emulator's assistance.
-The fine-tuned adapter is then returned to the model owner, who plugs it into the full model to create an adapted foundation model for users. 
-Offsite-tuning preserves both parties' privacy and is more computationally efficient than existing fine-tuning methods that require full model weights.
-We demonstrate the effectiveness of offsite-tuning on various large language and vision foundation models.
-We show that offsite-tuned foundation models can achieve comparable accuracy as full model fine-tuning while being privacy-preserving and efficient, with a 6.5x speedup and a 5.6x memory reduction.
+Transfer learning is important for foundation mod- els to adapt to downstream tasks. However, many foundation models are proprietary, so users must share their data with model owners to fine-tune the models, which is costly and raise privacy concerns. Moreover, fine-tuning large founda- tion models is computation-intensive and imprac- tical for most downstream users. In this pa- per, we propose Offsite-Tuning, an efficient and privacy-preserving transfer learning framework that can adapt billion-parameter foundation mod- els to downstream data without access to the full model. In offsite-tuning, the model owner sends a light-weight adapter and a lossy compressed emulator to the data owner, who then fine-tunes the adapter on the downstream data with the emu- lator’s assistance. The fine-tuned adapter is then returned to the model owner, who plugs it into the full model to create an adapted foundation model. Offsite-tuning is computationally efficient and better preserves both parties’ privacy than the existing fine-tuning methods that require access to the full model weights. We demonstrate the effectiveness of offsite-tuning on various large language and vision foundation models. Offsite- tuning can achieve comparable accuracy as full model fine-tuning while being privacy-preserving and efficient, achieving 6.5× speedup and 5.6× memory reduction.
 
 ## Setup
 
@@ -23,4 +15,4 @@ pip install transformers accelerate datasets evaluate wandb scikit-learn scipy t
 pip install lm-eval
 
 python setup.py develop
-```
+```