You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have used the repo to evaluate LLaVA and InternVL2 model, both met the same problem, if I gen_kwargs.pop("some_key", None) in previous requests, then in latter requests before executing that line of code, that key in gen_kwargs will already been popped. In general, the first two or three or more requests work fine, popping the key out of previous gen_kwargs won't affect the latter ones, but then at some point, all the gen_kwargs will have that key already been popped before exectuing that line of code.
To be pointed, the key is specified through cli commands, not written in the yaml file. generation_kwargs specified through cli, these settings will be used over set parameters in yaml tasks.
I don't know the logic of generating the requests, but I think this is a bug. Currently I use copy.deepcopy to fix this.
The text was updated successfully, but these errors were encountered:
I have used the repo to evaluate LLaVA and InternVL2 model, both met the same problem, if I
gen_kwargs.pop("some_key", None)
in previous requests, then in latter requests before executing that line of code, that key in gen_kwargs will already been popped. In general, the first two or three or more requests work fine, popping the key out of previous gen_kwargs won't affect the latter ones, but then at some point, all the gen_kwargs will have that key already been popped before exectuing that line of code.To be pointed, the key is specified through cli commands, not written in the yaml file. generation_kwargs specified through cli, these settings will be used over set parameters in yaml tasks.
I don't know the logic of generating the requests, but I think this is a bug. Currently I use copy.deepcopy to fix this.
The text was updated successfully, but these errors were encountered: