Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MoveNet singlepose-lightning yield weird result on with Vulkan backend on IREE. #19001

Open
RechieKho opened this issue Nov 3, 2024 · 3 comments
Labels
bug 🐞 Something isn't working codegen/spirv SPIR-V code generation compiler backend integrations/tensorflow TensorFlow model import and conversion

Comments

@RechieKho
Copy link
Contributor

What happened?

Hello, I'm the author of IREE.gd, a port of IREE to Godot game engine. Recently, I tried to use a pose detection model, specifically singlepose-lightning-int8 from MoveNet. It works like a charm on Metal backend but it is a total disaster on Vulkan backend.

This image is generated on Metal backend running on MacOS
Image

This image is generated on Vulkan backend running on Windows.
Image

The red dots are visual representation of the output coordinate resulted from running the model in IREE.

iree-gd/iree.gd#96

Steps to reproduce your issue

No response

What component(s) does this issue relate to?

Runtime

Version information

I'm using this git commit of IREE bb542eee65fa0a498963df1f2ee2f205a3dd8bd0.

IREE (https://iree.dev):
  IREE compiler version 20241031.1064 @ 2ec9017bb1de7d5ba552c1f016326088864b9cf9
  LLVM version 20.0.0git
  Optimized build

Additional context

No response

@RechieKho RechieKho added the bug 🐞 Something isn't working label Nov 3, 2024
@RechieKho
Copy link
Contributor Author

This is the TFLite, MLIR, VMFB and dump from iree-dump-module.

singlepose_lightning_int8.zip

@ScottTodd
Copy link
Member

Aside: I still want us to add coverage for TFLite models to iree-test-suites: iree-org/iree-test-suites#5. We should have a good process for adding test cases and validating them across configurations (CPU/Vulkan/Metal/etc.) when reports like this come in.

@ScottTodd ScottTodd added integrations/tensorflow TensorFlow model import and conversion codegen/spirv SPIR-V code generation compiler backend labels Nov 4, 2024
@fire
Copy link

fire commented Nov 4, 2024

If it helps the singlepose-thunder works. https://www.youtube.com/watch?v=FhTPeNDZlVI

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug 🐞 Something isn't working codegen/spirv SPIR-V code generation compiler backend integrations/tensorflow TensorFlow model import and conversion
Projects
None yet
Development

No branches or pull requests

3 participants