Constraint on weights and bias

[CatYukino]

Hi,

I'm new here so this may be a very easy question. Is there any way to restrict the weights and bias in the whole training process, like being nonnegative all the time? I just find in the doc that I can give a constraint on initial weights and bias, while I wonder if it is feasible in the whole process.

Thanks!

[avik-pal]

I am not very up to date with the literature here, but typically, these constraints can be applied using a reparameterization. For example, you can have unconstrained weights but apply an exp transform before feeding it into your model.