Wheel support for linux aarch64 [arm64]

[odidev]

Summary
Installing azure-uamqp-python on aarch64 via pip using command "pip3 install azure-uamqp-python" tries to build wheel from source code

Problem description
azure-uamqp-python doesn't have wheel for aarch64 on PyPI repository. So, while installing azure-uamqp-python via pip on aarch64, pip builds wheel for same resulting in it takes more time to install azure-uamqp-python. Making wheel available for aarch64 will benefit aarch64 users by minimizing azure-uamqp-python installation time.

Expected Output
Pip should be able to download azure-uamqp-python wheel from PyPI repository rather than building it from source code.

@azure-uamqp-python-team, please let me know if I can help you building wheel/uploading to PyPI repository. I am curious to make azure-uamqp-python wheel available for aarch64. It will be a great opportunity for me to work with you.

[yunhaoling]

hey @odidev , apologize for keeping you waiting for such a long time.

If I'm understanding it correctly, aarch64 is a subset of the ARM architecture, so the question here is to provide pre-built wheels for the ARM architecture (ARM32/64). And your suggestion sounds reasonable to me.

I'll bring team's attention to this topic of multi-arch wheel support and let you know how we wanna approach this.
cc: @lmazuel @annatisch

[odidev]

@yunhaoling, did you get any suggestion form the team on multi-arch wheel support?

[odidev]

Manylinux2014 ensures that the wheel is built against the AArch64 v8.0 specification. All AArch64 platforms in the v8 series maintain backward compatibility with v8.0. So a binary built for v8.0 can run on any v8 platform. Enabling aarch64 wheel support will benefit arm v8 series users by reducing installation time.

@lmazuel, @annatisch, @yunhaoling can you please let me know if you are interested in releasing aarch64 wheel support.

[yunhaoling]

hey @odidev, we would be happy to add support, but we first need to investigate how to manully build the whls on the the AArch64 platform.

basically to determine whether we need new VM agents with ARM architects, what're the supported python version, do we need new docker images for the build etc.

It would be nice if you could tell us more about the machine spec/os images you use (links, references, downloads etc.). 😊

[odidev]

Hi @yunhaoling , I tried building aarch64 wheels in manylinux2014_aarch64 docker container on my local aarch64 machine by using similar script as " build_many_linux.sh". Attached the script build_many_linux_aarch64.txt for your reference. The wheels are getting generate successfully.

If you don't have aarch64 machine, you can try the steps on travis-ci.com. Please let me know if anything else is required from my side.

[odidev]

@yunhaoling, please let me know if anything is required from my side. Thanks.

[odidev]

@yunhaoling, please let me know if I can help you building/uploading aarch64 wheels. Thanks

[yunhaoling]

hey @odidev , thanks for checking.

Apologize for missing the thread here -- I have been working on some urgent bug fixing issues these days.
But the arm64 wheel support is always on my mind, I'll probably have some time to look next week.

Our plan in general on this is to

1. Build the wheels locally by the information you provided on our dev boxes (thanks again)
2. To investigate how to integrate the extra build process with our internal devops system

Thanks for your patience and understanding in advance.

[odidev]

@yunhaoling, Could you please let me know if there is any update regarding this?

[yunhaoling]

hey @odidev , sincerely apologize for the late response. I was trying to use qemu to simulate a ARM 64 machine but the progress is slow.

On the other side, instead of supporting multiple platforms by building/compiling wheels, our team endeavors to get a pure python implementation of the amqp library which could help us get rid of this situation. (work is happening on the branch pyproto

I believe the pure python version is the long-term way we want to go and could resolve your issue.

[robcowie]

Hi. Is the decision of the maintainers not to provide pre-compiled wheels for aarch64?

I can certainly understand why if you are aiming to move to a pure python implementation but right now it causes difficulty when installing in potentially contained environments (i.e. a docker image without build tools) on arm hardware (i.e. new Apple gear).

[yunhaoling]

hey @robcowie , we understand that the aarch64 support is wanted and we have put efforts into how it could be supported. We succeeded to manually build uamqp wheel on a M1 Apple Mac without getting too much trouble -- install cmake first and then run pip3 install uamqp --no-binary :all worked.

However, we depend on internal Azure Devops VMs to build binary wheels, previously there was no ARM64 VMs being supported which means we just can't build and ship ARM64 wheels. The good news is that I learn last week that internally the ARM64 support VMs (Windows and Linux) are going through previews, and I would love to dogfood it and see how it goes.

[robcowie]

Thanks for the info @yunhaoling

[davidcroda]

Is this still on the roadmap?

[yunhaoling]

hey @davidcroda , yes, it's still on our track. We're still monitoring the azure devops agent support waiting for its public support.

are you looking for the uamqp library itself or you're using azure-eventhubs/azure-servicebus python sdk on aarch 64(arm64)?

the reason for asking you the question is because we're also working on pure python based amqp library which is being publicly developed in this branch: https://github.com/Azure/azure-uamqp-python/tree/pyproto.

And we've chosen to start the alpha of pure python amqp library as a vendored package within our Azure EventHubs and ServiceBus SDKs while we get that early feedback before we publish it as a standalone package.

You are very welcome to install and try out from the repo if you're seeking for amqp library or the azure-eventhub alpha SDK if you are trying to use azure-eventhub sdk.

We would appreciate any and all feedback!

[samgre]

Any updates?

[kashifkhan]

Hi all, I wanted to provide an update on this issue:

• We are going to be releasing a new version of the service bus client this week that will be using the new python based AMQP library. The event hub client library was released in January 2023 and is using the same python based library underneath. This will help folks who want to use them on aarch64 environments etc.
• Going forward with the releases of these two client libraries, uamqp will be phased out and active development will be reduced. Our focus on features, bug fixes etc. will be on the new python based library.
• If there is enough interest in the community, we will start planning and prioritizing efforts toward releasing a standalone version of the AMQP library.

thank you 🙂

[sugawarayss]

@kashifkhan
hi, I want to install uamqp via azure-iot-hub on MacOS 14.4.1, but I ran into this problem. How can I solve it? Or are there any alternatives?