使用案例指令评估LLaMA-7b模型在MMLU和C-Eval数据集上的性能时评测结果为空 #590
Unanswered
liujunfei678
asked this question in
Q&A
Replies: 3 comments 2 replies
-
Please check the log in *.out file |
Beta Was this translation helpful? Give feedback.
2 replies
-
您好,您遇到的这个问题解决了吗? |
Beta Was this translation helpful? Give feedback.
0 replies
-
我也遇到了相同的问题 |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
案例:
按照上述步骤确保OpenCompass正确安装并准备好数据集后,您可以使用以下命令评估LLaMA-7b模型在MMLU和C-Eval数据集上的性能:
python run.py --models hf_llama_7b --datasets mmlu_ppl ceval_ppl
结果:
![image](https://private-user-images.githubusercontent.com/134382914/282708559-7e39d88c-1d57-4937-84ed-ec7be4a31c5c.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Mzg5NDIwOTksIm5iZiI6MTczODk0MTc5OSwicGF0aCI6Ii8xMzQzODI5MTQvMjgyNzA4NTU5LTdlMzlkODhjLTFkNTctNDkzNy04NGVkLWVjN2JlNGEzMWM1Yy5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUwMjA3JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MDIwN1QxNTIzMTlaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT0wOTJkZmNhMDNjNDc2ZTkzOTYzODFjNTkzZjk3ZWNmODY1ZWRjMTM5N2Y3MmIzM2U4ZjA3ZmQ0NWMzZDU4NzFhJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.mQcjPyBdTIbvMZajUb4dDRIq1SR9ubGGtiYD3jnbmMA)
过程中存在的报错:
11/14 15:44:18 - OpenCompass - ERROR - /home/liujunfei/opencompass/opencompass/runners/base.py - summarize - 63 - OpenICLInfer[llama-7b-hf/lukaemon_mmlu_professional_law_0] failed with code 1
11/14 15:44:18 - OpenCompass - ERROR - /home/liujunfei/opencompass/opencompass/runners/base.py - summarize - 63 - OpenICLInfer[llama-7b-hf/lukaemon_mmlu_professional_law_1] failed with code 1
我的问题:
我跟着opencompass的安装指南一步步操作下来,不知道为什么出现了** 评测结果为空**,且出现了一些我不理解的报错消息。
Beta Was this translation helpful? Give feedback.
All reactions