Is it possible to use simple multi head self attention on bevformer encoder?

[jhkwag970]

Hello, Thank you for your work.

I am wondering is there any experiments that simple self attention is used for temporal self attention layer? instead of deformable self-attention?

I know based on Deforamble DETR, results should be better for deformable attention, but I am wondering how much superiority those deformalbe self attention can have compared to simple multi head self attention.

Thank you