RAdam vs AdamW with warmup

[Alxaline]

Hello,

Thank you for your excellent work. You have changed the core of the optimizer in ranger 2021 by AdamW, considering a warm-up instead of RAdam. Why the consideration of this choice? Was there any performance improvement according to your benchmark? Is there any others consideration than ease of implementation or excessive step size during the first iterations?

Best regards,