请问支持transformer模型吗？我尝试着使用vx-delegate 来推理whisper的tflite模型，但是总是报错

[jayyang-zigbang]

我使用了多个whisper的tflite模型, 在cpu模式下都能正常推理，但是换成 vx delegate就会报错

[jayyang-zigbang]

https://colab.research.google.com/drive/1CElRlLNbe9dRYpmu58jkbtu5lUKhMPRS?usp=sharing#scrollTo=dLkMa_36PgW-
我是用上面的代码转换得到的whisper tflite模型

[sunshinemyson]

@jayyang-zigbang ,

Sorry for late reply.

We are able to support transformer model. According to your log, you failed in the low-level driver, could you share which OpenVX Driver in your test? and please share log with "export VSI_NN_LOG_LEVEL=5" and " export VIV_VX_DEBUG_LEVEL=1 "

Thanks

[jayyang-zigbang]

@sunshinemyson
hi, firstly so thank you for your reply.
I was on the verge of giving up on the project,😭 but your reply has kindled a glimmer of hope within me. 😄

As you requested, I have exported some low-level logs. Please help me review them.
i.mx8plus_testwhisper_usingTflite-vx-delegate_error_log.txt

Regarding your inquiry about the OpenVX Driver, were you referring to the version of the OpenVX Driver used in my test? Unfortunately, I'm not sure how to check the version of the OpenVX Driver.

By the way, I conducted tests on i.MX 8M Plus EVK using TFLite models. Is there anything specific you'd like me to do or any questions you have about these tests?

thank you again.

[jayyang-zigbang]

I wondered if there was something wrong with the tflite-vx-delegate. I also tried to rebulid tflite-vx-delegate from source. But i got fail.

[sunshinemyson]

I wondered if there was something wrong with the tflite-vx-delegate. I also tried to rebulid tflite-vx-delegate from source. But i got fail.

I can not figure out the issue from this snapshot. Maybe you can check with newer gcc version.

I review the log, i need to forward this to internal debug. Will post latest status here.

[sunshinemyson]

@jayyang-zigbang jay，

We tried model by ourself, the model cannot run with tflite CPU reference implementation. Did you tried with CPU?

Thanks

[jayyang-zigbang]

hi @sunshinemyson , i am not sure which tflite file you tried. i will upload my tflite file. its works well on i.MX 8M Plus EVK cpu mode.
https://drive.google.com/drive/folders/17f-FZ1rdh1x9Bh8rm2tL6E9v4BvBdfpC?usp=sharing
if you have question just let me know it. thank you again.

[sunshinemyson]

@jayyang-zigbang ,

Thanks, We will try it again with your model.

Thanks

[robert-kalmar]

Hi @jayyang-zigbang, what version of BSP you are using? I suppose you use the Yocto Linux from NXP.

[jayyang-zigbang]

Hi @robert-kalmar, I only ran these tflite models on the original i.MX 8M Plus EVK linux. I didn't try to install any other linux os(BSP) on the i.MX 8M Plus EVK board.

[robert-kalmar]

@jayyang-zigbang , copy me please the output of uname -a. Based on this I can identify the BSP version.

[jayyang-zigbang]

@jayyang-zigbang , copy me please the output of uname -a. Based on this I can identify the BSP version.

Thank you for your prompt response. I'll check the BSP version tomorrow as I didn't bring the development board home with me. While we're on the topic, have you had a chance to test my Whisper TFLite models on the NPU? How do they perform on the NXP NPU? Or did you any experience for running transformer architecture on nxp npu?

[jayyang-zigbang]

@robert-kalmar hi this is the output of uname -a ==> Linux imx8mp-alpha 5.15.71+g15429ab64ee9 #1 SMP PREEMPT Wed Apr 12 06:36:33 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux

@jayyang-zigbang , copy me please the output of uname -a. Based on this I can identify the BSP version.