A Tensorflow Implementation of SMU: SMOOTH ACTIVATION FUNCTION FOR DEEP NETWORKS USING SMOOTHING MAXIMUM TECHNIQUE
https://arxiv.org/abs/2111.04682
Please check https://github.com/iFe1er/SMU_pytorch for pytorch implementation.
Tested with Tensorflow 2.x. For Tensorflow 1, simply repalce tf.compat.v1.get_variable with tf.get_variable would do the trick.
@ARTICLE{2021arXiv211104682B, author = {{Biswas}, Koushik and {Kumar}, Sandeep and {Banerjee}, Shilpak and {Pandey}, Ashish Kumar}, title = "{SMU: smooth activation function for deep networks using smoothing maximum technique}", journal = {arXiv e-prints}, keywords = {Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Computer Science - Computer Vision and Pattern Recognition, Computer Science - Neural and Evolutionary Computing}, year = 2021, month = nov, eid = {arXiv:2111.04682}, pages = {arXiv:2111.04682}, archivePrefix = {arXiv}, eprint = {2111.04682}, primaryClass = {cs.LG}, adsurl = {https://ui.adsabs.harvard.edu/abs/2021arXiv211104682B}, adsnote = {Provided by the SAO/NASA Astrophysics Data System} }