How is OneTrainer able to train StableCascade Lora ?

[WillIsback]

Hello all,

I am new to Ai training, and recently I have train a StableCascade LoRA model using the StableCascade train script. I run training on my rtx2080ti and I notice a few difference between training with OneTrainer and the train script from StabilityAi.

I can only train the stage_C_1B model otherwise Cuda out of memory.

How is OneTrainer able to train Lora with the priorCascade wich i understood from other discussion is stage_C_3B. Is it because it use quantization 4bits or something alike ?

With StabilityAi training script there is no Epoch parameters, How does epochs implementation works. As i understood it is like one training run. Is it able to perform computation of large model by dividing one large training run by a multitude of little run ?

I get that the training of stable cascade come with a pecular architecture such as wurstcore, did you manage on top of that to add ways to compute larger model on little gpu ? If so could you please explain to me a little then I will research about how to compute larger model on little gpu ?

While training with train script I had a very hard time and even once went through overfitting where my model was reproducing noise. I think I will write a jupyter about my training journey as I face many difficulties. Please share with me the little things you learn along the way as I am eager to improve myself on this subject.

Thank you all for your time. Cheers to @Nerogar, you rock dude It must be a huge amount of time and it is so nice of you to share it with us, much appreciate.

William