Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

voyage zero-shot checkmark #2099

Open
Muennighoff opened this issue Feb 18, 2025 · 1 comment
Open

voyage zero-shot checkmark #2099

Muennighoff opened this issue Feb 18, 2025 · 1 comment

Comments

@Muennighoff
Copy link
Contributor

If the training data is unknown, there should probably be no checkmark for zero-shot?

Image
@x-tabdeveloping
Copy link
Collaborator

As far as I know, we have had discussions about this, and in cases, where the training data can't be disclosed, we take the model maker's word on whether they have trained on any of the MTEB task datasets.
Voyage hasn't told us specifically what they trained on, but they have told us that they did not train on these datasets, while with voyage-3-m-exp they told us that they did.
I think this is fair play as long as the companies stay transparent and accountable.

Though this could be up for debate.

cc @KennethEnevoldsen

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants