First shot at proper transfer learning

[zouharvi]

Train at opus.yaml, then copy trained model and run a few epochs on base.yaml.

[pixelneo]

By base.yaml, you mean wmt19.yaml in current naming?

[zouharvi]

Yes.

[pixelneo]

ok, transfer learning is done: make tqt runs it, results are still bad.
branch transfer

[zouharvi]

Ok, there must be something wrong. F1 7% is way under the constant classifier score.

[pixelneo]

So, I've tried:

• modifying learning rate and number of epochs for both models
• train on wmt, save, load, train again
• switch quetch for nuqe.

None of this helps. When using nuqe the drop in F1 is not that significant but stil considerable.
Also, I've noticed, when loading a saved model, following info is printed:
[kiwi.data.utils load_vocabularies_to_fields:126] Loaded vocabularies from runs/quetch/best_model.torch

which seems a little weird since I am not loading vocabularies.
I've tried to also explicitly load vocab from the vocab.torch file (and run again several settings - lr, epochs, ...) This did not help.

So basically, I don't know where to go from here other than searching for a bug in openkiwi.