uestion about --model_name and --pt_model_name arguments for running fine-tuning DPO code

[HIsu1231]

Hello,

I have reviewed the code associated with your paper on GitHub and am attempting to use it for fine-tuning. I have a couple of questions regarding the --model_name and --pt_model_name arguments in the execution command:

1. What value should be assigned to the --model_name argument? I am curious to know whether this argument specifies the architecture of the model or if it refers to a pre-trained model.

2. What is the purpose of the --pt_model_name argument? What model name should be entered for this argument to enable proper fine-tuning, and to replicate the experiments described in your paper?

If you could provide an example of the execution command or any additional documentation links, it would be greatly helpful for my study. Thank you for facilitating deeper research through your code. I’ll continue to explore the documentation while I await your response.

Thank you.

[Nish-19]

Hi @HIsu1231,

Thanks for your interest in our work.

--model_name is the name used for saving the model that gets trained.

--pt_model_name is the name of the pre-trained model (trained using standard fine-tuning) required for the DPO step.

I have updated the readme with some examples of commands to run.