Anyone had success in LORA training flux1? (not on an h100) #472
pinballelectronica
started this conversation in
General
Replies: 1 comment
-
|
everything you say doesn't make sense with 2x4090 :D even 8gb gpus are able to train FLUX dev model and starting from 16 GB you can train with best quality rtx 4090 can train very fast and 2x trains even faster - lora rtx 4090 can train a decent speed full fine tuning as well |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Curious what settings people have used to actually get a practical training speed on a consumer GPU. Floats, optimizers, ranks, alphas. My go-to is Prodigy but due to the ramp up time that may not be ideal. My 2x4090's are no match for float16 across the board. 95 steps a NIGHT at very conservative values :) I
hear nf4 produces very poor output so. Trying to find a sweet spot. Thank you.
This image took ~7 minutes at FP16.
Beta Was this translation helpful? Give feedback.
All reactions