'DiffusionTransformer' object has no attribute 'cf_predict_start'

[songxueXS]

Hi author, thanks for sharing your inspiring work!
I'm trying to make re-implementation of Improved VQ-Diffusion. I have the problem.

Traceback (most recent call last):
File "/env/anaconda3-pytorch1.7-python3.7-cuda11/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 19, in _wrap
fn(i, *args)
File "/code/VQ-Diffusion-Improved_VQ-Diffusion/image_synthesis/distributed/launch.py", line 93, in distributed_worker
fn(local_rank, *args)
File "/code/VQ-Diffusion-Improved_VQ-Diffusion/train.py", line 168, in main_worker
solver.train()
File "/code/VQ-Diffusion-Improved_VQ-Diffusion/image_synthesis/engine/solver.py", line 546, in train
self.train_epoch()
File "/code/VQ-Diffusion-Improved_VQ-Diffusion/image_synthesis/engine/solver.py", line 463, in train_epoch
self.sample(batch, phase='train', step_type='iteration')
File "/code/VQ-Diffusion-Improved_VQ-Diffusion/image_synthesis/engine/solver.py", line 198, in sample
samples = model.sample(batch=batch, step=self.last_iter)
File "/env/anaconda3-pytorch1.7-python3.7-cuda11/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 26, in decorate_context
return func(*args, **kwargs)
File "/code/VQ-Diffusion-Improved_VQ-Diffusion/image_synthesis/modeling/models/dalle.py", line 303, in sample
**kwargs)
File "/code/VQ-Diffusion-Improved_VQ-Diffusion/image_synthesis/modeling/transformers/diffusion_transformer.py", line 592, in sample
log_z, sampled = self.p_sample(log_z, cond_emb, t, sampled, self.n_sample[diffusion_index]) # log_z is log_onehot
File "/env/anaconda3-pytorch1.7-python3.7-cuda11/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 26, in decorate_context
return func(*args, **kwargs)
File "/code/VQ-Diffusion-Improved_VQ-Diffusion/image_synthesis/modeling/transformers/diffusion_transformer.py", line 278, in p_sample
model_log_prob, log_x_recon = self.p_pred(log_x, cond_emb, t)
File "/code/VQ-Diffusion-Improved_VQ-Diffusion/image_synthesis/modeling/transformers/diffusion_transformer.py", line 267, in p_pred
log_x_recon = self.cf_predict_start(log_x, cond_emb, t)
File "/env/anaconda3-pytorch1.7-python3.7-cuda11/lib/python3.7/site-packages/torch/nn/modules/module.py", line 779, in getattr
type(self).name, name))
torch.nn.modules.module.ModuleAttributeError: 'DiffusionTransformer' object has no attribute 'cf_predict_start'

[koutilya-pnvr]

VQ-Diffusion/image_synthesis/modeling/transformers/diffusion_transformer.py

Line 267 in fe79083

log_x_recon = self.cf_predict_start(log_x, cond_emb, t)

More precisely, this is the code line from the Improved_VQ-Diffusion branch. The cf_predict_start function is not defined in the DiffusionTransformer class. Is it the same as the one from the dalle class?

VQ-Diffusion/image_synthesis/modeling/models/dalle.py

Line 170 in fe79083

def cf_predict_start(log_x_t, cond_emb, t):

[songxueXS]

VQ-Diffusion/image_synthesis/modeling/transformers/diffusion_transformer.py

Line 267 in fe79083

log_x_recon = self.cf_predict_start(log_x, cond_emb, t)

More precisely, this is the code line from the Improved_VQ-Diffusion branch. The cf_predict_start function is not defined in the DiffusionTransformer class. Is it the same as the one from the dalle class?

VQ-Diffusion/image_synthesis/modeling/models/dalle.py

Line 170 in fe79083

def cf_predict_start(log_x_t, cond_emb, t):

Thank you very much for your reply, but I find they are not exactly the same and 'def cf_predict_start(log_x_t, cond_emb, t)' depends on many parameters. Can you give a full version?

[tzco]

Sorry for late reply. The cf_predict_start function is defined in generate_content in dalle.py for classifier-free sampling, but while training we don't need classifier-free sampling and it is not executed. By adding a placeholder in diffusion_transformer.py it should be solved:

def cf_predict_start(self, log_x_t, cond_emb, t):
    return self.predict_start(log_x_t, cond_emb, t)