what is the exact meaning of iterating in the parameters ?

[Oussamab21]

Hi , I just wanted to know what is the exact effect of these instructions:

for p in list(model.parameters()):
if hasattr(p,'org'):
p.data.copy_(p.org)

for p in list(model.parameters()):
if hasattr(p,'org'):
p.org.copy_(p.data.clamp_(-1,1))

[Ashokvardhan]

@Oussamab21 : The first for loop is to ensure that when you take a gradient step it's on the real weight parameters. However before this line p.data.copy_(p.org), the weights are actually binary because binary weights are needed for forward and backward passes. Once the backward pass is done using binary weights, we actually want to update the real weights and hence we copy the real weights which were temporarily stored in self.weight.org here

The second for loop is to ensure that weights are bounded in [-1,1].

[Oussamab21]

Thank you if you only wants to binarize the weights is there any thing to change ?