Cannot run step 2 due to potential version mismatch.

[sweetice]

Hi! Thanks for sharing us with such a great repository!
I do step 2 after step 1. However, some error occurs.

The following is the error information.

Traceback (most recent call last):
  File "/mnt/c/Users/qiang/Documents/GitHub/RLPHF/gpt4_annotate/run.py", line 106, in <module>
    main(args)
  File "/mnt/c/Users/qiang/Documents/GitHub/RLPHF/gpt4_annotate/run.py", line 81, in main
    annotator = PairwiseAutoAnnotator(annotators_config = args.annotators,
  File "/mnt/c/Users/qiang/Documents/GitHub/alpaca_farm/src/alpaca_farm/auto_annotations/eval.py", line 170, in __init__
    super().__init__(
  File "/home/mirror/anaconda3/envs/py310/lib/python3.10/site-packages/alpaca_eval/annotators/pairwise_evaluator.py", line 51, in __init__
    super().__init__(*args, **kwargs, primary_keys=self.input_keys + self.output_keys)
  File "/home/mirror/anaconda3/envs/py310/lib/python3.10/site-packages/alpaca_eval/annotators/base.py", line 436, in __init__
    super().__init__(*args, **kwargs)
  File "/home/mirror/anaconda3/envs/py310/lib/python3.10/site-packages/alpaca_eval/annotators/base.py", line 116, in __init__
    self.annotators = self._initialize_annotators()
  File "/home/mirror/anaconda3/envs/py310/lib/python3.10/site-packages/alpaca_eval/annotators/base.py", line 210, in _initialize_annotators
    return {
  File "/home/mirror/anaconda3/envs/py310/lib/python3.10/site-packages/alpaca_eval/annotators/base.py", line 211, in <dictcomp>
    name: self.SingleAnnotator(
  File "/mnt/c/Users/qiang/Documents/GitHub/alpaca_farm/src/alpaca_farm/auto_annotations/eval.py", line 186, in __init__
    super().__init__(*args, **kwargs)
  File "/home/mirror/anaconda3/envs/py310/lib/python3.10/site-packages/alpaca_eval/annotators/pairwise_evaluator.py", line 371, in __init__
    super().__init__(
TypeError: SingleAnnotator.__init__() got an unexpected keyword argument 'prompt_templates'

It seems that the version of alpaca_farm and alpaca_eval is a mismatch.

Could you please share with us the versions of alpaca_farm and alpaca_eval you use? Or could you please update your code to match the current alpaca_farm and alpaca_eval versions?

Some information may help you.

pip list | grep alpaca
alpaca_eval              0.5.2
alpaca-farm              0.1.12

Thx!

[FSSLI]

Hello, how did you handle the basic model in your first step