Skip to content

Issues: modelscope/evalscope

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

21 Open 88 Closed
21 Open 88 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Too little data to calculate quantiles! perf
#252 opened Dec 18, 2024 by balcklive
2 tasks
1
0.8.1版本下遇见的humaneval数据集测评问题
#250 opened Dec 18, 2024 by WjMessi1
9 tasks
cannot import name 'EvalBackend' from 'evalscope.utils' native
#247 opened Dec 17, 2024 by maoulee
1 of 10 tasks
2
使用 local_vllm 压测一直超时 perf
#243 opened Dec 15, 2024 by divinerapier
3 tasks done
@Yunnglin
10
问答评估 native
#233 opened Dec 9, 2024 by lizhen-lizhen
@Yunnglin
1
使用opencompass评测后端结果和官网结构不一样 opencompass
#229 opened Dec 5, 2024 by yosure
4 tasks
@wangxingjun778
1
当模型推理结果为空时会报错 native
#228 opened Dec 3, 2024 by ChavesLiu
@Yunnglin
4
评估越来越慢，gpu利用率低 native
#227 opened Dec 3, 2024 by ChavesLiu
2 tasks done
@wangxingjun778
1
采用 Openai server 测试，能够产生 inference 结果，但是 eval 输出结果为空 opencompass
#208 opened Nov 22, 2024 by HaoWuSR
9 tasks
@wangxingjun778
2
bbh测试集之前可以测，现在测不了了 bug Something isn't working native
#202 opened Nov 14, 2024 by charliedream1
2
使用python代码如何统一指定离线data目录？ documentation Improvements or additions to documentation
#201 opened Nov 13, 2024 by charliedream1
opencompass数据集中safety板块数据集支持 enhancement New feature or request
#197 opened Nov 12, 2024 by A-Zi-cong-xiao-jiu-hen-ke-ai
2
评估提示模板 enhancement New feature or request
#195 opened Nov 12, 2024 by lizhen-lizhen
1
OpenCompass打印的测试集列表和网址给出的不一致，导致无法测试，比如cmmlu enhancement New feature or request
#191 opened Nov 9, 2024 by charliedream1
@wangxingjun778
11
能否给个longwriter和toolbench使用openai接口测试的例子 documentation Improvements or additions to documentation
#189 opened Nov 9, 2024 by charliedream1
1
输出结果没有分数 bug Something isn't working opencompass
#181 opened Nov 8, 2024 by Leo20100307
9 tasks
@wangxingjun778
6
evalscope perf 测试sglang 部署的openai api server 无法输出结果 perf question Further information is requested
#128 opened Sep 12, 2024 by hetian127
@wangxingjun778
4
请问，评估支持使用昇腾910NPU嘛？ good first issue Good for newcomers
#123 opened Sep 9, 2024 by yiyayieryo
4
Baseline模型对比模式结果错误
#118 opened Aug 29, 2024 by stay-leave
2
请教一下，如何使用openai兼容格式的大模型作为评估模型来进行两模型在自定义数据集上的评估任务呢？
#115 opened Aug 26, 2024 by EvilCalf
3
model_id为local ckpt路径时，infer template映射和解析出现问题
#24 opened Mar 26, 2024 by wangxingjun778
ProTip! Updated in the last three days: updated:>2024-12-15.