Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle normalizations and regularizations more flexibly #191

Open
ndiamant opened this issue Mar 25, 2020 · 0 comments
Open

Handle normalizations and regularizations more flexibly #191

ndiamant opened this issue Mar 25, 2020 · 0 comments
Labels
enhancement New feature or request

Comments

@ndiamant
Copy link
Contributor

ndiamant commented Mar 25, 2020

What
Model normalizers and regularizers placed automatically in correct order

Why
Normalizers and regularizers are currently placed in fixed positions, which prevents the use of some useful techniques. For example, L2 regularization is impossible because right now because regularization layer comes after the activation instead of the layer.

How
Dictionary mapping regularization enum or string to position in block.

Acceptance Criteria

  • Dictionary mapping regularization enum or string to position in block
  • New regularizations + normalizations properly used (especially L2, layer norm)
  • Command line allows multiple regularizations
  • Command line fails early on nonexistent regularizations and normalizations
@ndiamant ndiamant added the enhancement New feature or request label Mar 25, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant