Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support new task mmworld #269

Merged
merged 2 commits into from
Sep 22, 2024
Merged

Support new task mmworld #269

merged 2 commits into from
Sep 22, 2024

Conversation

jkooy
Copy link
Contributor

@jkooy jkooy commented Sep 21, 2024

Before you open a pull-request, please check if a similar issue already exists or has been closed before.

When you open a pull-request, please be sure to include the following

  • A descriptive title: [xxx] XXXX
  • A detailed description

Thank you for your contributions!

Hi authors,

Thanks for your awesome work for creating such a useful tools!
This pull-request add a new task MMWorld https://mmworld-bench.github.io/, a new benchmark for multi-discipline, multi-faceted multimodal video understanding.

Copy link
Collaborator

@kcz358 kcz358 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR. Can you share a screenshot of a successful evaluation on this dataset and fix the lint issue? After that would be good to merge.

To auto fix the lint, you can use pre-commit and simply run pre-commit run --all-files

@jkooy
Copy link
Contributor Author

jkooy commented Sep 21, 2024

Thank you for the PR. Can you share a screenshot of a successful evaluation on this dataset and fix the lint issue? After that would be good to merge.

To auto fix the lint, you can use pre-commit and simply run pre-commit run --all-files

Hi, here is the evaluation tested on my end. I have also fixed the lint issue

image

@kcz358
Copy link
Collaborator

kcz358 commented Sep 22, 2024

Thank you for your contribution @jkooy , LGTM!

@Luodian Luodian merged commit 0bae7fa into EvolvingLMMs-Lab:main Sep 22, 2024
1 check passed
KairuiHu pushed a commit that referenced this pull request Sep 23, 2024
* support new task mmworld

* Apply linting fixes
KairuiHu pushed a commit that referenced this pull request Oct 24, 2024
* support new task mmworld

* Apply linting fixes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants