Can it downgrade for running on V100?

[forrest-lam]

I had run webui.py, but errors occured on generating 4 views.
otImplementedError: No operator found for memory_efficient_attention_forward with inputs:
query : shape=(10, 6144, 1, 64) (torch.bfloat16)
key : shape=(10, 6144, 1, 64) (torch.bfloat16)
value : shape=(10, 6144, 1, 64) (torch.bfloat16)
attn_bias : <class 'NoneType'>
p : 0.0
decoderF is not supported because:
attn_bias type is <class 'NoneType'>
bf16 is only supported on A100+ GPUs
[email protected] is not supported because:
requires device with capability > (8, 0) but your GPU has capability (7, 0) (too old)
bf16 is only supported on A100+ GPUs
cutlassF is not supported because:
bf16 is only supported on A100+ GPUs
smallkF is not supported because:
max(query.shape[-1] != value.shape[-1]) > 32
dtype=torch.bfloat16 (supported: {torch.float32})
bf16 is only supported on A100+ GPUs
unsupported embed per head: 64

[realisticdreamer114514]

https://developer.nvidia.com/cuda-gpus
The "requires device with capability" part points to this, which means it requires at least RTX 30 or 40-series GPUs (and their enterprise counterparts). Need to add settings that support older cards and apply it automatically once it detects your GPU type.

(I would also want to see if it is possible to run on consumer-grade GPUs)

[zjp-shadow]

You can try turning off this option memory_efficient_attention_forward to run this code. V100's memory is enough.

[realisticdreamer114514]

You can try turning off this option memory_efficient_attention_forward to run this code. V100's memory is enough.

Where can we find this option?

[forrest-lam]

You can try turning off this option memory_efficient_attention_forward to run this code. V100's memory is enough.

Is it the flag enable_memory_efficient_attention in 3D_Stage/configs/infer.yaml? But it occurs in two places in this file, one in image_tokenizer and the other in backbone. Which one should I turn off?

[realisticdreamer114514]

You can try turning off this option memory_efficient_attention_forward to run this code. V100's memory is enough.

Is it the flag enable_memory_efficient_attention in 3D_Stage/configs/infer.yaml? But it occurs in two places in this file, one in image_tokenizer and the other in backbone. Which one should I turn off?

Update: I turned both off, it doesn't help, the same error appears.

If the dev didn't just leave while refusing to elaborate further, we would have a easier time.