AdapterBERT is a project that utilizes the BERT model with adapter layers for fine-tuning on specific downstream tasks. This project is an implementation of the paper: Parameter-Efficient Transfer Learning for NLP, Houlsby [Google], ICML 2019.
To set up the project, follow these steps:
-
Clone the repository:
git clone https://github.com/nikhil-chigali/AdapterBERT.git
-
Navigate to the project directory:
cd AdapterBERT -
Install Poetry:
pip install poetry
-
Set up the Poetry environment:
poetry install --no-root
Update the training hyperparameters and settings as needed in the constants.py file.
To train the model, run the train.py script. Here's an example command:
python train.pyBefore running any inferences, update the path to the model checkpoint CKPT_PATH in constants.py
To make predictions on custom input prompt using the trained model, run the predict.py script:
python predict.pyI would like to acknowledge the following repositories and papers used as reference for this project:
- bert_adapter repo by @strawberrypie
- transformers repo by @huggingface
- Parameter-Efficient Transfer Learning for NLP, Houlsby [Google], ICML 2019.