You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, this framework only supports BLOOM models with DeepSpeed
However, various results have shown that OPT-175B is better than BLOOM (citation needed).
Ideally, we would like this framework to support OPT-175B as well.
The main work would be in the form of loading OPT-175B in an huggingface model and sharding with deep-speed. This would probably require a machine with loads of RAM (but hopefully without the need for 8 GPUs)
I've started an issue in the original repo for this huggingface/transformers-bloom-inference#39
The text was updated successfully, but these errors were encountered:
Currently, this framework only supports BLOOM models with DeepSpeed
However, various results have shown that OPT-175B is better than BLOOM (citation needed).
Ideally, we would like this framework to support OPT-175B as well.
The main work would be in the form of loading OPT-175B in an huggingface model and sharding with deep-speed. This would probably require a machine with loads of RAM (but hopefully without the need for 8 GPUs)
I've started an issue in the original repo for this huggingface/transformers-bloom-inference#39
The text was updated successfully, but these errors were encountered: