cifar10-reproduce

[Reciphon]

I want to reproduce the result show in the paper on cifar10-dataset.
While std_path converges well, csg_path is not really working. The accuracy on csg_path is very low.
Could you please provide more details about cifar10-dataset? Including the hyper-parameters and so on.
Looking forward to your reply!

[Harry-Up]

Hello. I encounter the same problem. This problem occurs after the warm-up epochs, and the loss becomes extremely unstable. If I reduce the learning rate of the Adam optimizer to 1e-6, there will still fluctuate violently. Even if I do not reduce it or only reduce it to 1e-4, nan will appear directly. I don't know yet whether it is the problem of the algorithm itself or the code. Do you have any ideas?

[Harry-Up]

Well, the learning rate of reg is also the key point, and the result is ok if it is reduced.