scaled_dot_product_attention output is different from memory_efficient_attention

[aenoca]

❓ Questions and Help

I have been playing with DINOv2-small and trying to replicate some of its outputs by rebuilding it from scratch. I noticed that the results I got when running with scaled_dot_product_attention versus with memory_efficient_attention were NOT the same.

I wonder if anyone can check this out.

[sebhtml]

Can you provide more details including:

• small code example that reproduce the problem,
• expected result
• actual result
  ?