Gregory LeMasurier and Mojtaba Talaei Khoei
In this project we compare a randomly initialized PEGASUS model to a pretrained PEGASUS model for abstractive summarization. We trained and fine-tuned our models on the cnn_dailymail dataset. We found that the pretrained model performed much better than our randomly initialized model, as seen by the Rouge scores.
Install all dependencies in the requirements.txt.
NOTE: Please read the instructions to install bitsandbytes
properly as it depends on the cudatoolkit version!
Train a base model: python random_model/random_model.py
Train a pretrained model: python full_model/full_model.py
Modify cell two to include the proper path and file name to your model config and bin file.
Run all cells in demo.ipynb to run the demo using model you trained.
Go to the local url output by the 3rd block, for us it was: http://127.0.0.1:7860/
Paste text you would like to sumarize in the left most gradio block.
Press submit and the summary will be displayed in the rightmost block.
https://wandb.ai/glemasurier/PegasusSummarization?workspace=user-glemasurier