multiple-choice question?

[sailfish009]

Hello, instead of asking the model to describe the image in detail, I want to ask them to answer by selecting 1 option from several choices. Questions that worked well in llava onevision MLLM are not working in Aria. For example, if I ask them to answer the situation in image P by selecting from A, B, C, D, E, if the correct answer is A, they will answer with an additional comment about the situation A. I want to make sure that the additional answer should never be included, so that no words other than A are added.

[LiJunnan1992]

You may want to try adding this sentence to your question:
"Answer with the option's letter from the given choices directly."

[sailfish009]

@LiJunnan1992 Hi, I tried adding the question format you mentioned, but it didn't improve the model's answers. The model still answers randomly without choosing from a list of answers.

[Coobiw]

Hi, you may refer to https://github.com/rhymes-ai/Aria/tree/main/examples/nextqa. This is a multiple-choice question-answering dataset. Training, inference, instruction format and option extraction are all included. Hope it helpful for you~

[sailfish009]

@Coobiw Thanks for the answer, I took a quick look at the questions in nextqa and it looks like the model is sensitive to the format of the question.