Question about sample_sparse_structure

[kevinhuangxf]

Thanks for the excellent work!

I have a question about the sample_sparse_structure function. If it samples difference size of coordinates from the image encoded latents, how can this operation be applied for multiple batch size during training ?

[JeffreyXiang]

Hi,
Batched attention with different sequence length can be efficiently handled with modern attention implementation like flash-attn and xformers. See the code here for more details.