Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Porting OPT-175B to this framework #10

Open
CoderPat opened this issue Dec 19, 2022 · 0 comments
Open

Porting OPT-175B to this framework #10

CoderPat opened this issue Dec 19, 2022 · 0 comments

Comments

@CoderPat
Copy link
Collaborator

Currently, this framework only supports BLOOM models with DeepSpeed
However, various results have shown that OPT-175B is better than BLOOM (citation needed).

Ideally, we would like this framework to support OPT-175B as well.
The main work would be in the form of loading OPT-175B in an huggingface model and sharding with deep-speed. This would probably require a machine with loads of RAM (but hopefully without the need for 8 GPUs)
I've started an issue in the original repo for this huggingface/transformers-bloom-inference#39

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant