Skip to content

Issues: embeddings-benchmark/mteb

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

118 Open 406 Closed
118 Open 406 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

leaderboard 2.0: custom urls
#1518 opened Nov 27, 2024 by Muennighoff
leaderboard 2.0: Performance on subtasks (Bright retrieval)
#1517 opened Nov 27, 2024 by KennethEnevoldsen
leaderboard v2.0: disagreement in scores between v2.0 and v1.0
#1516 opened Nov 27, 2024 by KennethEnevoldsen
leaderboard 2.0: Add missing models
#1515 opened Nov 27, 2024 by KennethEnevoldsen
leaderboard 2.0: Some task <> benchmark associations seem wrong
#1514 opened Nov 27, 2024 by KennethEnevoldsen
Odd column breaks on the leaderboard e.g. the S of STS lands on a newline
#1513 opened Nov 27, 2024 by KennethEnevoldsen
Add description of metrics to the leaderboard
#1512 opened Nov 27, 2024 by KennethEnevoldsen
@x-tabdeveloping @KennethEnevoldsen
Need help with integrating the model jxm/cde-small-v1
#1511 opened Nov 27, 2024 by YashDThapliyal
3
Sort task information (either by name or task type) leaderboard issues related to the leaderboard
#1510 opened Nov 27, 2024 by KennethEnevoldsen
Remove nan scores on leaderboard leaderboard issues related to the leaderboard
#1509 opened Nov 27, 2024 by KennethEnevoldsen
Add minishlab models good first issue Good for newcomers new-model Questions related to adding a new model to the benchmark
#1503 opened Nov 26, 2024 by KennethEnevoldsen
1
Add mixedbread-ai/mxbai-embed-xsmall-v1 and mixedbread-ai/mxbai-embed-2d-large-v1
#1502 opened Nov 26, 2024 by KennethEnevoldsen
Add voyage-3 and voyage-3-lite new-model Questions related to adding a new model to the benchmark
#1501 opened Nov 26, 2024 by KennethEnevoldsen
Merge main into v2.0.0
#1500 opened Nov 26, 2024 by KennethEnevoldsen
@KennethEnevoldsen @Samoed
1
Investigate/Fix ClimateFever
#1498 opened Nov 25, 2024 by Muennighoff
Expose get_model in README.md
#1497 opened Nov 25, 2024 by Muennighoff
1
Cannot reproduce results on text classification benchmark.
#1490 opened Nov 23, 2024 by tsWen0309
15
Add Language Families
#1485 opened Nov 22, 2024 by isaac-chung
1
@isaac-chung
Add text-embedding-005 new-model Questions related to adding a new model to the benchmark
#1480 opened Nov 21, 2024 by emerzon
Add descriptive_stats to all tasks help wanted Extra attention is needed
#1475 opened Nov 19, 2024 by Samoed
@imenelydiaker
12
add google/siglip-so400m-patch16-256-i18n
#1474 opened Nov 19, 2024 by SaitejaUtpala
1
Can't load some datasets because of datasets 3.0 bug Something isn't working
#1465 opened Nov 16, 2024 by Samoed
2
Add NanoBEIR new-dataset
#1464 opened Nov 16, 2024 by Muennighoff
1
Importing mteb is slow good first issue Good for newcomers
#1463 opened Nov 16, 2024 by Muennighoff
@Napuh
7
[mieb] add support for LLM2CLIP mieb The image extension of MTEB
#1458 opened Nov 14, 2024 by SaitejaUtpala
ProTip! Updated in the last three days: updated:>2024-11-24.