You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As a user of prompt tuning, I want to be able to leverage multiple GPUs at train time!
Discussion
Extends #175 to leverage PyTorch FSDP; there may be some refactoring to consolidate with fine tuning FSDP configuration / dataset stuff, which may be part of this story, or left to a follow up. We will decide once we see how Peft / trainer integration goes.
Description
As a user of prompt tuning, I want to be able to leverage multiple GPUs at train time!
Discussion
Extends #175 to leverage PyTorch FSDP; there may be some refactoring to consolidate with fine tuning FSDP configuration / dataset stuff, which may be part of this story, or left to a follow up. We will decide once we see how Peft / trainer integration goes.
Acceptance Criteria
The text was updated successfully, but these errors were encountered: