ONNX export

[ionymikler]

Hi @falcon-xu

I am looking into how to export the model into ONNX, with the intention of benchmarking its performance in different machines and with potential optimization techniques like quantization and others...
I see there is some minimal code in the repo that suggests that you attempted to do this (the DeiTOnnxConfig class in configuration_deit.py fx. , but I can't seem to find any scripts actually using this.
Any recommendations on how to proceed?

I understand from reading about the way pytorch does onnx export that the default process using torch.jit.trace() does not capture the dynamics of the model, so I guess an alternative that does so is needed. Have you played with this?

[falcon-xu]

Hi @ionymikler,

Currently, we do not export the model into ONNX. As you mentioned, using torch.jit.trace() doesn’t capture the full dynamics of the model, particularly when the model has control flow or other dynamic operations. For that, I guess you can use torch.onnx.export() with dynamic axes or explore exporting through torch.jit.script(), which handles more complex models with dynamic behavior.

If you want further guidance on the specifics of exporting with ONNX or need help with integrating this into your workflow, the community of ONNX may help you.

Best regards,
Falcon