No speedup and memory saving on CIFAR10

[guangzhili]

I have played around with CIFAR10 and also done a bit benchmark. It seems BinOp does not have noticeable effect on model size and inference speed compared to NIN model without BinOp. I have tested both on CPU and GPU. I thought the saved model nin.pth.tar would shrink, and the inference would speed up significantly. Do I miss something? Does anyone have this issue? Thanks.

[fenollp]

It’s because BinOp is still 32 or 16 bits. There is no packing optimization here. Sad.

[guangzhili]

@fenollp Thanks for your reply. What would you suggest to do if I would like to achieve the binary optimization? modify PyTorch core?

[zhuyinheng]

@guangzhili useful discussion here here
And a GPU kernel based on TensorFlow can be found here
But unfortunately, the acceleration is NOT significant. As for compression, I think it's easy to implement.

[jiecaoyu]

@guangzhili An XNOR operation kernel is required to get an acceleration.

I am implementing the kernels as part of my research project. I will release the code after I get the paper published somewhere.

[guangzhili]

Thank you @cow8 That's very helpful!

[guangzhili]

@jiecaoyu Sounds great. Good luck with the paper.

[mjczyt]

@guangzhili look forward to see your paper ,good luck

[Paul0629]

My NIN model without BinOp is 946.7k while model with BinOp is 3.9M. That's weird.@jiecaoyu