Question on (5.2.2 Continual-Finetuning of WeLore Compressed Models)

[sugarandgugu]

hi,thanks for your great work. I have some question on the section "Continual-Finetuning of WeLore Compressed Models"
Q1: Firstly, you perform rank reduction in Llama7B in your paper, then using lora and welore to continual-finetuning. I read your code, performing rank reduction will product two matrix 'U,V', then using lora and welore, which means produce four matrix for 'U,V'? Meanwhile, using welore to finetune, which means you guys perform rank reduction again?
Q2: Secondly, you use lora to finetune, your parameters is not shown, such as lora rank.

[sugarandgugu]

New Question: How to merge the UV matrix to the pretrain model（llama2-7b-base）?

[sugarandgugu]

I've been waiting.

[zuininanHHH9527]

I've been waiting.

感觉是编故事的，文章看着奇奇怪怪的，method也不写清楚，要不是半成品要不是编故事