Can we apply modified sampling algorithm to the problems presented in the paper?

[kevin-on]

Hello, I have been reading your paper and had a few questions that I thought you might be able to help with.

In the paper, you argue that the proposed algorithm can solve problems that cannot be solved with the previous modified sampling algorithm. However, to maximize c(x, y) in the modified sampling algorithm, couldn't we just calculate the gradient of c(x', y) and adjust each sampling step, where x' is the predicted x0 in each timestep? Is there some critical issue with this sampling algorithm that I am not aware of?

I would greatly appreciate any insights you could provide on this matter. Thank you for your time and for the valuable contributions you have made to the field.

[qubewx]

I am also interested in how you could explain your question in detail.

[AlexGraikos]

Hello and thanks for the constructive comments,

I think that what you are proposing is similar to the idea presented here https://arxiv.org/pdf/2111.14818.pdf where the authors perform gradient steps using \hat{x_0} to also minimize the distance of the sampled image to a target CLIP embedding.

So yes, what you are proposing is definitely possible!