From 1f4ad975aaed67132d61ddfaa9d2891e95102260 Mon Sep 17 00:00:00 2001
From: Song <3068011+songhan@users.noreply.github.com>
Date: Wed, 8 Feb 2023 22:22:24 -0500
Subject: [PATCH] Update README.md

---
 README.md | 14 +++-----------
 1 file changed, 3 insertions(+), 11 deletions(-)

diff --git a/README.md b/README.md
index c1b8182..fc49080 100644
--- a/README.md
+++ b/README.md
@@ -1,17 +1,9 @@
-# offsite-tuning
+# Offsite-Tuning
 The official implementation of Offsite-Tuning.
 
 ## Abstract
 
-Foundation models have become increasingly popular due to their general performance on downstream tasks.
-Although some foundation models can make zero-shot predictions, transfer learning (e.g. fine-tuning) is still important for adaption to specific downstream tasks.
-However, many foundation models are proprietary, so users must share their data with model owners to fine-tune the models, which is costly and raise privacy concerns. Moreover, fine-tuning foundation models is computation-intensive and impractical for most downstream users.
-In this paper, we propose Offsite-Tuning, a privacy-preserving and efficient transfer learning framework that can adapt foundation models to downstream data without full model access.
-In offsite-tuning, the model owner sends a light-weight adapter and a lossy compressed emulator to the data owner, who then fine-tunes the adapter on the downstream data with the emulator's assistance.
-The fine-tuned adapter is then returned to the model owner, who plugs it into the full model to create an adapted foundation model for users. 
-Offsite-tuning preserves both parties' privacy and is more computationally efficient than existing fine-tuning methods that require full model weights.
-We demonstrate the effectiveness of offsite-tuning on various large language and vision foundation models.
-We show that offsite-tuned foundation models can achieve comparable accuracy as full model fine-tuning while being privacy-preserving and efficient, with a 6.5x speedup and a 5.6x memory reduction.
+Transfer learning is important for foundation mod- els to adapt to downstream tasks. However, many foundation models are proprietary, so users must share their data with model owners to fine-tune the models, which is costly and raise privacy concerns. Moreover, fine-tuning large founda- tion models is computation-intensive and imprac- tical for most downstream users. In this pa- per, we propose Offsite-Tuning, an efficient and privacy-preserving transfer learning framework that can adapt billion-parameter foundation mod- els to downstream data without access to the full model. In offsite-tuning, the model owner sends a light-weight adapter and a lossy compressed emulator to the data owner, who then fine-tunes the adapter on the downstream data with the emu- lator’s assistance. The fine-tuned adapter is then returned to the model owner, who plugs it into the full model to create an adapted foundation model. Offsite-tuning is computationally efficient and better preserves both parties’ privacy than the existing fine-tuning methods that require access to the full model weights. We demonstrate the effectiveness of offsite-tuning on various large language and vision foundation models. Offsite- tuning can achieve comparable accuracy as full model fine-tuning while being privacy-preserving and efficient, achieving 6.5× speedup and 5.6× memory reduction.
 
 ## Setup
 
@@ -23,4 +15,4 @@ pip install transformers accelerate datasets evaluate wandb scikit-learn scipy t
 pip install lm-eval
 
 python setup.py develop
-```
\ No newline at end of file
+```