Update to pytorch-0.4

[sylvchev]

Using pytorch-0.4.1 to run this code, I encountered several problems. I'm creating this pull request to propose a solution to these issues.
One blocking issue is related with generate_fast() that raise a RuntimeError: the number of sizes provided (1) must be greater or equal to the number of dimensions in the tensor (2)
I also modified to code to avoid warnings:

• nn.functional.tanh and sigmoid are deprecated in favor of torch.tanh and sigmoid
• scalar should be accessed with item()
• Variable are deprecated in favor of requires_grad args

Unfortunately, I could not verify that these modifications are functional on pytorch < 0.4. I'll try to install pytorch 0.3 on another system and see if it is backward compatible.

Thank you very much for this nice code, I was a huge help to understand WaveNet! I hope I could contribute to this project.

[bobchennan]

Works for me on 0.4

[gecko17k]

The code in demo.ipynb requires saber dataset.
Where is this dataset?

I switched to the only available dataset in this repo: bach/chaconne, but then I get an error message that ends with :

" RuntimeError: Expected object of type torch.FloatTensor but found type torch.cuda.FloatTensor for argument #2 'weight' "
Here is the error message in full:

RuntimeError Traceback (most recent call last)
in ()
11 tic = time.time()
12 trainer.train(batch_size=8,
---> 13 epochs=20)
14 toc = time.time()
15 print('Training took {} seconds.'.format(toc - tic))

/media/disk1/vince/PythonCodes/waveNet/torch/pytorch-wavenet/wavenet_training.py in train(self, batch_size, epochs, continue_training_at_step)
68 x = Variable(x.type(self.dtype))
69 target = Variable(target.view(-1).type(self.ltype))
---> 70 output = self.model(x)
71 loss = F.cross_entropy(output.squeeze(), target.squeeze())
72 self.optimizer.zero_grad()

~/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py in call(self, *input, **kwargs)
489 result = self._slow_forward(*input, **kwargs)
490 else:
--> 491 result = self.forward(*input, **kwargs)
492 for hook in self._forward_hooks.values():
493 hook_result = hook(self, input, result)

/media/disk1/vince/PythonCodes/waveNet/torch/pytorch-wavenet/wavenet_model.py in forward(self, input)
186 def forward(self, input):
187 x = self.wavenet(input,
--> 188 dilation_func=self.wavenet_dilate)
189
190 # reshape output

/media/disk1/vince/PythonCodes/waveNet/torch/pytorch-wavenet/wavenet_model.py in wavenet(self, input, dilation_func)
125 def wavenet(self, input, dilation_func):
126
--> 127 x = self.start_conv(input)
128 skip = 0
129

~/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py in call(self, *input, **kwargs)
489 result = self._slow_forward(*input, **kwargs)
490 else:
--> 491 result = self.forward(*input, **kwargs)
492 for hook in self._forward_hooks.values():
493 hook_result = hook(self, input, result)

~/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py in forward(self, input)
174 def forward(self, input):
175 return F.conv1d(input, self.weight, self.bias, self.stride,
--> 176 self.padding, self.dilation, self.groups)
177
178

RuntimeError: Expected object of type torch.FloatTensor but found type torch.cuda.FloatTensor for argument #2 'weight'

Thanks.

[gecko17k]

Turns out the solution is to put the arguments
dtype=dtype, ltype=ltype
into the function WaveNetTrainer(), like so:

trainer = WavenetTrainer(model=model,
                           dataset=data,
                           lr=0.001,
                           weight_decay=0.0,
                           gradient_clipping=None,
                           snapshot_path='snapshots',
#                            snapshot_name='saber_model',                                                    
                           snapshot_name='bach_model',
                           snapshot_interval=100000,
                           dtype=dtype,
                           ltype=ltype, 
                           )

Credit to @ianmktu.

[pathnirvana]

Also you need to move the model to cuda
if use_cuda: model.cuda()