Fine-tuning on already fine tuned model stuck at "copying training files"

[charliekocsis]

Dear Tommi and all,

I'm very excited about this development and the professional work involved with it! Thank you!

I have trained a Hungarian base model that I downloaded from the program.

I then decided to further fine-tune it with more TMX memory.

However when I start to load the next TMX, the progress is stuck at "Copying training files."

It never finishes that. If I close and reopen the program it says "Ok."

But it isn't ok for sure...

If I take the same TMX that bugs the already fine-tuned model and use it on the base mode, it is running.

I can't find error log or anything like that. Any idea or suggestion why this could be? I'm using the latest pre-release 1.2.4.0. I also tried prior to that with the actual released version, same result.

Thank you.

Best Regards,
Charlie

[TommiNieminen]

Currently you can't fine-tune models that have already been fine-tuned. Unfortunately the user interface currently doesn't block it, it just hangs during the process. I should either make it possible to do repeated fine-tuning or to just remove the option. I'm not sure what repeated fine-tuning will do for model quality, it might degrade it eventually, that's why I'm a bit wary of it.

[charliekocsis]

Thank you Tommi for the response! I understand your concern. Perhaps if you have a version that can retrain multiple times, I could give you feedback as to it's quality improvement or degradation. Maybe if we made it so that it can be re-trained with further data but has a way to add "priority" or importance of data. Meaning, if just general text being trained, the data fed to it has lower importance as opposed to terminology data of a specific company or something like that. This is just an idea. Thought to mention, in case you find it interesting.