Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Model] support minimonkey model #257

Merged
merged 2 commits into from
Sep 16, 2024
Merged

Conversation

white2018
Copy link
Contributor

Before you open a pull-request, please check if a similar issue already exists or has been closed before.

When you open a pull-request, please be sure to include the following

  • A descriptive title: [xxx] XXXX
  • A detailed description.

Thank you for your contributions!

This PR intends to enhance the functionality of lmms-eval by supporting MiniMonkey model in Monkey series, which is not covered in latest version.

Copy link
Collaborator

@kcz358 kcz358 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @white2018 , I think most of the changes look good. Can you share a screenshot of the evaluation result using this model? Then I think it will be good to merge.

@white2018
Copy link
Contributor Author

Hi @white2018 , I think most of the changes look good. Can you share a screenshot of the evaluation result using this model? Then I think it will be good to merge.

Hi @kcz358 , I run this evaluation code with mmstar dataset. Here is the screeshot as follows:
image

@kcz358
Copy link
Collaborator

kcz358 commented Sep 15, 2024

@white2018 , the result looks good, can you do a final lint to pass the format checking? Current the workflow has failed. You can do it by

python3 -m pip install pre-commit
pre-commit install
pre-commit run --all-files

And simply commit the formatted files would be good. Thank you!

@white2018
Copy link
Contributor Author

white2018 commented Sep 15, 2024

@white2018 , the result looks good, can you do a final lint to pass the format checking? Current the workflow has failed. You can do it by

python3 -m pip install pre-commit
pre-commit install
pre-commit run --all-files

And simply commit the formatted files would be good. Thank you!

@kcz358 All right! It is done. Thanks a lot

@kcz358
Copy link
Collaborator

kcz358 commented Sep 16, 2024

Thank you for your contribution. Merging this PR now.

@kcz358 kcz358 merged commit a2a881c into EvolvingLMMs-Lab:main Sep 16, 2024
1 check passed
@white2018 white2018 deleted the minimonkey branch September 16, 2024 11:08
@white2018 white2018 restored the minimonkey branch September 16, 2024 11:10
@white2018 white2018 deleted the minimonkey branch September 16, 2024 11:13
@white2018 white2018 restored the minimonkey branch September 16, 2024 11:23
@white2018 white2018 deleted the minimonkey branch September 16, 2024 11:23
@white2018 white2018 restored the minimonkey branch September 16, 2024 11:32
@white2018 white2018 deleted the minimonkey branch September 16, 2024 11:41
@white2018 white2018 restored the minimonkey branch September 17, 2024 07:49
@white2018 white2018 deleted the minimonkey branch September 17, 2024 08:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants