Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add the SEEDBench 2 Plus dataset and minor change in dataset docs.
Key Changes:
Add SEEDBench 2 Plus Dataset:
Added the SEEDBench 2 Plus dataset proposed in the paper, which already implemented in VLMEvalKit.
Rename Existing SEEDBench 2 Task Names:
Changed the task names in the existing SEEDBench 2 dataset from hyphens (-) to underscores (_) to maintain consistency with other datasets. (seedbench-2 -> seenbench_2)
Add Descriptions for seedbench 2 plus:
Added descriptions for seedbench 2 plus
I have completed tests for SEEDBench 2 Plus benchmark on several models with this code change, but please leave a comment if further review is needed.
The data uploaded to the datasets repository was created with the following code:
If the datasets repository should be managed by lmms-lab, I would appreciate comments on the necessary steps to handle that aspect.
Thank you.
Before you open a pull-request, please check if a similar issue already exists or has been closed before.
When you open a pull-request, please be sure to include the following
Thank you for your contributions!