Any plans for supporting Copilot+PC NPU acceleration (e.g. via ONNX) for Python?

[AndreasKunar]

I use SK on the new Copilot+PCs with Windows on ARM (WoA). While llama.cpp runs great on the Snapdragon X CPUs, and even comparable to base Apple Silicon+GPU with their new 2-3x accelerated Q4_0_4_8 quantization, it does not (yet) support its GPU or NPU. Ollama needs custom builds to run on WoA. Semantic Kernel .net supports ONNX.

Any plans to support ONNX from Python and not just .net, so that the arm64 Copilot+PCs can use their NPU from Python / Semantic Kernel?

[moonbox3]

Hi @AndreasKunar, thanks for your question. Yes, we have plans to support the ONNX AI connector in SK Python. We have a work item tracking this in our backlog. This work should be picked up in a few sprints from now.

[AndreasKunar]

Thank you!