Support for MultiScaleDeformableAttn module

[Thaslim]

I am trying to profile BEVFormer which has MultiScaleDeformableAttn module,
current tool throws error:
NotImplementedError: this Node MultiScaleDeformableAttnTRT2-MultiScaleDeformableAttnTRT2_2825 has no value_infer
Is there any workaround/ plan to support Deformable attention layer.
I am mainly looking to get list of OPs and MAC/ Param counts.

[ThanatosShinji]

You should profile this model with custom-registered nodes, refer this

[Thaslim]

Hi
I get the following error when trying the example mentioned using @NODE_REGISTRY
When I try to import NODEPROFILER_REGISTRY, it throws an error

/home/thasleem.tajudeen/miniconda3/envs/cramnet/lib/python3.9/site-packages/onnx_tool/node.py:2511: UserWarning: node MultiScaleDeformableAttnTRT2 is not registed for profiling, return 0 Macs and 0 params as default. Use NODEPROFILER_REGISTRY to register your profiler for this node.
  warnings.warn(f'node {n.op_type} is not registed for profiling, return 0 Macs and 0 params as default. '
Node Init Time Elapsed 0.007455110549926758
Tensor Init Time Elapsed 0.11277580261230469
IO Tensor Init Time Elapsed 0.0007681846618652344
Traceback (most recent call last):
  File "/home/thasleem.tajudeen/enntools-workspace/BFCP/bfcp_head/bev.py", line 44, in <module>
    bevformer()
  File "/home/thasleem.tajudeen/enntools-workspace/BFCP/bfcp_head/bev.py", line 37, in bevformer
    m = onnx_tool.Model(file,{'verbose':True,"constant_folding":True})
  File "/home/thasleem.tajudeen/miniconda3/envs/cramnet/lib/python3.9/site-packages/onnx_tool/model.py", line 26, in __init__
    self.graph = Graph(m.graph, self.cfg)
  File "/home/thasleem.tajudeen/miniconda3/envs/cramnet/lib/python3.9/site-packages/onnx_tool/graph.py", line 184, in __init__
    self.__constant_search__(self.cfg.constant_folding)
  File "/home/thasleem.tajudeen/miniconda3/envs/cramnet/lib/python3.9/site-packages/onnx_tool/graph.py", line 290, in __constant_search__
    this_node.value_infer(itensors, otensors)
  File "/home/thasleem.tajudeen/miniconda3/envs/cramnet/lib/python3.9/site-packages/onnx_tool/node.py", line 158, in value_infer
    raise NotImplementedError(f'this Node {self.op_type}-{self.name} has no value_infer')
NotImplementedError: this Node MultiScaleDeformableAttnTRT2-MultiScaleDeformableAttnTRT2_2825 has no value_infer

[ThanatosShinji]

@Thaslim It seems that you used a different ONNX model. In my example, the TRT operator is MultiScaleDeformableAttnTRT, but you got MultiScaleDeformableAttnTRT2.

[Thaslim]

Hi @ThanatosShinji ,The ONNX model I have uses this TRT plugin

[ThanatosShinji]

how to get this model?