How can I produce the visualized image after ShuntendTransformer?

[MingfangDeng]

I want to see the result( a image)after ShuntendTransformer,but the type of output is tensor,and the channel is not 3,I don't know how to
acquire the image after ShuntendTransformer?
I will appreciate if you can help me,thank you very much.

[OliverRensu]

Hi We follow these codes https://github.com/facebookresearch/dino/blob/main/visualize_attention.py https://github.com/hila-chefer/Transformer-Explainability

[rayleizhu]

1. As far as I know, each heatmap is corresponding to a query point. so what's the query point for the visualization in Figure 3?
2. Which level of attention block did you use for visualization?

[go-ahead-maker]

I find that in DINO, the default backbone is ViT or DeiT, which uses the CLS token to represent the whole image-level information. So the visualization result is corresponding to the CLS token.
But Shuntend does not use CLS token, so It may be difficult to adopt the attention map to reflect the RoI of the whole image, and Grad-CAM may be the alternative.