Transfer learning

[jekim5418]

Hello, may I ask for transfer learning?

Among the transfer learning settings presented in the paper, when the teacher-student pair is ResNet32x4-ShuffleV1, the performance when I reproduce, does not match.
After training student with baseline and vanilla KD, I tried transfer learning on tiny-imagenet with the trained.
However, the baseline did not exceed 33, and the vanilla KD did not exceed 31.

How can I achieve the performance presented in the paper?

[Zzzzz1]

Follow the setting reported on the paper. In particular, the weight decay should be set as 0.0 and the backbone(and the BN modules) should be fixed.