[help] Where are the generated adapters_name saved? Can they be downloaded and used post-training?

[LissetheGarcia]

No description provided.

[LissetheGarcia]

I just want to corraborate that after running this command:

moe_peft.trainer.save_adapter_weight(model=model, config=train_config, path="dir_to_save")

we will get the ".json" and ".bin" files in a folder which path can be used in

model, config = MixLoraModelForCausalLM.from_pretrained("adapter_name_folder")

is this correct?

[mikecovlee]

Yes. The weights output by MoE-PEFT is fully compatible with this repository.

[mikecovlee]

adapter_name will not be saved. Then you can load it anywhere and any name else for any other purpose.

[LissetheGarcia]

Thank you! this definitely helped me :)