Merge pull request #443 from Sleort/patch-1

Slightly faster softplus
FluxML · Nov 18, 2022 · f7597d9 · f7597d9
2 parents 7087954 + 7eea8b8
commit f7597d9
Showing 1 changed file with 1 addition and 1 deletion.
diff --git a/src/activations.jl b/src/activations.jl
@@ -615,7 +615,7 @@ julia> softplus(16f0)
 16.0f0
 ```
 """
-softplus(x) = log1p(exp(-abs(x))) + max(x, 0)
+softplus(x) = log1p(exp(-abs(x))) + relu(x)
 
 """
     logcosh(x)