How do I target NPU for quantization?

[kleiti]

I can provide the execution provider like this:
config.StaticQuantConfig(calibration_data_reader=data_reader, quant_format=QuantFormat.QOperator, execution_provider="DmlExecutionProvider"), but there's no way to specify the device as NPU?

[mengniwang95]

Hi, if you use ONNX Runtime to run the quantized model on NPU, providing execution provider is enough.

[kleiti]

How does it select between GPU and NPU if using DmlExecutionProvider?

[mengniwang95]

Hi, since currently we only provide limited support and do limited test for DML EP, we don't expose many parameter about this EP.
You can check this PR of onnxrutnime about device selecting for DML EP microsoft/onnxruntime#17612.
Actually you can quantize the model first, and then select device during later inference.