Use onnx_tool to fix TensorRT's model issues

1.

[11/10/2022-20:33:00] [E] [TRT] ModelImporter.cpp:779: ERROR: builtin_op_importers.cpp:647 In function importConv:
[8] Assertion failed: inputs.at(2).is_weights() && "The bias tensor is required to be an initializer for the Conv operator."

[11/10/2022-22:33:27] [E] [TRT] ModelImporter.cpp:779: ERROR: builtin_op_importers.cpp:3608 In function importResize:
[8] Assertion failed: scales.is_weights() && "Resize scales must be an initializer!"

Root cause:
pytorch to ONNX conversion created Identity layer for bias tensor. like:

Solution:

#onnx-tool version >=0.3.2
python -m onnx_tool -m rm_iden -i raw.onnx -o removed.onnx

After:

Now TensorRT can execute this onnx model.

2.

[11/10/2022-22:36:12] [E] Static model does not take explicit shapes since the shape of inference tensors will be determined by the model itself

[11/10/2022-22:37:52] [W] Dynamic dimensions required for input: input, but no shapes were provided. Automatically overriding shape to: 1x3x1x1

Root cause:

ONNX model with fixed input shapes, like 1x3x128x128, you can't set the dynamic inputs with TensorRT. E.g.

trtexec --onnx=fixed.onnx --minShapes=input:1x3x4x4 --optShapes=input:1x3x16x16 --maxShapes=input:1x3x128x128

On the opposite side, you must set these shape ranges with a dynamic ONNX model. If you ignore the shape ranges, TensorRT will give you a warning about this, and set the dynamic part of input shapes to 1. Therefore, you can only infer this model with 1x3x1x1 as the input tensor's shape.

Solution:

#onnx-tool version >=0.3.2
python -m onnx_tool -m io_modify -i fixed.onnx -o dynamic.onnx -d input:1x3xhxw #from fixed input to dynamic input
python -m onnx_tool -m io_modify -i dynamic.onnx -o fixed.onnx -d input:1x3x128x128 #from dynamic input to fixed input

After:

#onnx-tool version >=0.3.2
trtexec --onnx=fixed.onnx
trtexec --onnx=dynamic.onnx --minShapes=input:1x3x4x4 --optShapes=input:1x3x16x16 --maxShapes=input:1x3x128x128