Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Packaging] Add wheels for Alpine Linux #18036

Open
asfimport opened this issue Jan 30, 2018 · 22 comments
Open

[Packaging] Add wheels for Alpine Linux #18036

asfimport opened this issue Jan 30, 2018 · 22 comments

Comments

@asfimport
Copy link
Collaborator

asfimport commented Jan 30, 2018

Currently when attempting to install pyarrow using pip on Alpine Linux you get the following error message from pip:

 

pip install pyarrow --verbose
Collecting pyarrow
  1 location(s) to search for versions of pyarrow:
  * https://pypi.python.org/simple/pyarrow/
  Getting page https://pypi.python.org/simple/pyarrow/
  Looking up "https://pypi.python.org/simple/pyarrow/" in the cache
  Current age based on date: 596
  Freshness lifetime from max-age: 600
  Freshness lifetime from request max-age: 600
  The response is "fresh", returning cached response
  600 > 596
  Analyzing links from page https://pypi.python.org/simple/pyarrow/
    Skipping link https://pypi.python.org/packages/03/fe/d3e86d9a534093f84ec6bb92c5285796feca7713f9328cc2b607ee9fc158/pyarrow-0.2.0-cp35-cp35m-manylinux1_x86_64.whl#md5=283d6d42277a07f724c0d944ff031c0c (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/06/e9/ac196752b306732afedf415805d327752bd85fb1e4517b97085129b5d02e/pyarrow-0.4.1-cp27-cp27mu-manylinux1_x86_64.whl#md5=884433983d1482e9eba7cdedd82201e5 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/0b/1c/c7e00871d85f091cbe4b71dd6ff2ce393b6e736d6defd806f571da87280c/pyarrow-0.5.0-cp36-cp36m-win_amd64.whl#md5=d7e3d8b9d17e7a964c058f879e11e733 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/0b/e8/e907b7e597981e488d60ea8554db0c6b55a4ddc01ad31bb18156f1dc1526/pyarrow-0.5.0.post2-cp34-cp34m-manylinux1_x86_64.whl#md5=9353e2bcfc77a2b40daa5d31cb9c5dac (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/0c/01/2e283b8fae727c4932a4335e2b1980a65c2ef754c69a7d97e39b0157627d/pyarrow-0.7.0-cp34-cp34m-manylinux1_x86_64.whl#md5=6d8ec243f77a382667b6f9b0aa434fd2 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/0c/19/805aa541740279bc8a198eeeb57509de5551f55f0cbc6371fa897bfc3245/pyarrow-0.8.0-cp35-cp35m-manylinux1_x86_64.whl#md5=382cb788fd740b0e25be3b305ab46142 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/0d/39/b0e21b10b53386f3dad906a8b734074cc0008c5af6a31d2e441569633d94/pyarrow-0.6.0-cp36-cp36m-manylinux1_x86_64.whl#md5=392930f4ace76ac65965258f5da99e9d (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/0f/22/97ba96f7178a52f257b45eac079d7a47dc4bc22d0961e828f10a76c254a7/pyarrow-0.4.1-cp35-cp35m-macosx_10_6_intel.whl#md5=96db8da8ee09952e62731ef8afd1f20d (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/15/5c/20192ab842b291d889f12f7013a5ac5c4416e231377024ad6823fc42a456/pyarrow-0.8.0-cp35-cp35m-win_amd64.whl#md5=8123173dc4905e7186ecf35ba180817a (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/20/b6/50f42a2dd53e0679ffe5fb74bdc745fcad3b5e0975e9ae356256c0370d06/pyarrow-0.7.1-cp35-cp35m-macosx_10_6_intel.whl#md5=5d06b3332b5bac0682d55f20ab2cb556 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/22/2f/7bf9225142d9db6e67e74cff8a18aa98514159cb5c96b15d15044db9ea5f/pyarrow-0.7.1-cp35-cp35m-win_amd64.whl#md5=111be7aac9a73210c2b1ae8e1e459819 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/23/60/f3db27c6a201994a5b1afb4f263afdfa22f5292380379900d7af325d679f/pyarrow-0.5.0-cp35-cp35m-win_amd64.whl#md5=cf45b4190ba1079cc2532c1a9fd09285 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/24/a3/9ebac24b3ad5ee0a95b8050b84299e988be265ee2fbb45d5a70cc61ab3ba/pyarrow-0.4.0-cp27-cp27m-macosx_10_6_intel.whl#md5=9bc1cb87ee62ef0b9a4f3d022545640f (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/26/02/f6af7a2377f627ea1e41108bcd6319230202d8c68c86aedbcb141d1c25cb/pyarrow-0.4.0-cp36-cp36m-macosx_10_6_intel.whl#md5=6135414a8f7f2f464d9a5187eea8dbde (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/29/05/0f4af49d02ae7685ef6821dc5b624e0cd88eeb1527146eb28a42fb675641/pyarrow-0.5.0-cp27-cp27m-macosx_10_6_intel.whl#md5=eb9efb75fa563dd5945403113341276a (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/2b/da/4f3b150c21ad8759fe66caa5e66fc13a9e489e26247b6c5490118fbebab7/pyarrow-0.3.0-cp36-cp36m-manylinux1_x86_64.whl#md5=4b6de1b69635b3039d1a666b9401f7ab (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/2d/2b/52d3f5828cf6253f36eed849e4b15e11a0d0ed258a9cd7f38c3867b02024/pyarrow-0.7.1-cp36-cp36m-manylinux1_x86_64.whl#md5=acea1613e2d9c491736e533bd68bf4ae (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/2d/cd/f53afc0849cbd276f1f5a889a08d8622d7cdd462ca60b62db63260004612/pyarrow-0.8.0-cp36-cp36m-win_amd64.whl#md5=628f380ddd348f8576559a791448f646 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/33/20/30dda14607785d5a1ae75b874d91a1071a6f474c1dafcc91c285a5a333b3/pyarrow-0.7.1-cp27-cp27mu-manylinux1_x86_64.whl#md5=8161b46f86696dea99ea8cc90637f3e1 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/35/f3/ee2af8aa68142d17aca5baa7fea2ed46a3696ffbd1aa3c01afba2ee21cb7/pyarrow-0.6.0-cp27-cp27mu-manylinux1_x86_64.whl#md5=0701ed9fe9b053e5f9d7b22054716eae (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/37/0e/3b28463f6ecf1fb183d246314c2352f9f06f91bcb5ebf9ba1f1796102c92/pyarrow-0.7.1-cp36-cp36m-win_amd64.whl#md5=4b2b7c58c21050e6413b75f84dc8d050 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/3f/54/c56471ff0bfc9bbdd458a3e4da35c9cba78df59bffbcb5af2d9069f756a0/pyarrow-0.8.0-cp36-cp36m-manylinux1_x86_64.whl#md5=b7f42d55455ab6daa61ee2991313c819 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/42/c9/119fe67523e71770f62bc681864d76a304babbcc215c6e9ccdadc6e57573/pyarrow-0.7.0-cp36-cp36m-macosx_10_6_intel.whl#md5=436b9d796b96336f667d123eb74ad1e9 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/52/b9/381dd068546109e310a14f5a197db26628d52c7598d2f03786a4f95ace74/pyarrow-0.4.1-cp36-cp36m-macosx_10_6_intel.whl#md5=6af77a81a66e9e32791241db90505d10 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/55/44/854bbcac02eb74106030e0d5c66497384381305db670ec068461e6733482/pyarrow-0.7.1-cp27-cp27m-macosx_10_6_intel.whl#md5=10829c1acba07101743427f5d52033d1 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/59/3e/db938e9ce067c69c57a6586d6d9735c4ac83e6e3890a6851048a1809a79d/pyarrow-0.4.0-cp34-cp34m-manylinux1_x86_64.whl#md5=7f0f0ba987c577777c760f3402050995 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/5c/26/4dbf4cf8ee0ebde967a062774a4385bc3a29c2f282c27ca82f6a7eb1a609/pyarrow-0.4.1-cp34-cp34m-manylinux1_x86_64.whl#md5=bc150c73bae15d798c4067049f3fb748 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/62/84/581a7edcfdaaaf304668b19d142ddec586bc25c9587a7da78b5d265c2965/pyarrow-0.7.1-cp34-cp34m-manylinux1_x86_64.whl#md5=6c7e5583cc9b86d405e59b24dfce8112 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/62/ad/c90216b2062cf0420c30f7dbada7271306c75e39a36b46875edc9ab60587/pyarrow-0.8.0-cp36-cp36m-macosx_10_6_intel.whl#md5=17ba8e082889559c362cc0c123b6aca6 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/66/23/e2fcc901452bda46e386dc4440c0a8fb2557941730748c8fa97cb101940b/pyarrow-0.7.0-cp36-cp36m-win_amd64.whl#md5=68c17074b240702f479547cccfab4bd7 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/69/92/28c1b9d274f077ab949a41c83b202c11fe4b57778b062c6253410e3b2239/pyarrow-0.4.0-cp36-cp36m-manylinux1_x86_64.whl#md5=5c5788016a8df45a74b61412113e426a (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/6c/d6/98b5fa5f5d8a7840e4d17b4af586248de35a4cd1e056120e4950e35bbf09/pyarrow-0.8.0-cp34-cp34m-manylinux1_x86_64.whl#md5=711036bbde129f971722bd9fe969577c (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/72/d4/59f03b8e639f83bb960e4676343e5e087898ab8999a80b1f2ed95e98d3a1/pyarrow-0.5.0-cp36-cp36m-macosx_10_6_intel.whl#md5=f2f397dd92808a9b186f4b6b1846f8f1 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/76/67/53192e7a70f9633f7d6c98d965e419d3c778a061aa096109a00506e97c84/pyarrow-0.6.0-cp27-cp27m-macosx_10_6_intel.whl#md5=b0450d6ad20a264fa44d1a8c67d1d75e (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/77/26/81ba3ecdfc23d151e8a762e3ea5a1b3ff2e89e3757a902bfef55c3a0096d/pyarrow-0.6.0-cp36-cp36m-win_amd64.whl#md5=6812881bd232804da121ae64e4f4f4de (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/7d/15/8357e9ed09acf2e30204a3533d9eb7a7798618a35269f01e0f0ddd4ccdd3/pyarrow-0.8.0-cp27-cp27m-macosx_10_6_intel.whl#md5=6ca7d3ac0b68d0afcb375bbb1eee6b2b (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/80/d0/3f79c879f16d2e75d9cbf351cd6463a0a6d7e45def4bf1ae0db12d424dc8/pyarrow-0.6.0-cp34-cp34m-manylinux1_x86_64.whl#md5=9cbcfed997c9adfa60b36bdf8f6d04cf (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/83/b8/7fd3b9369e7454ec4953804bd9ce0946203a6477f80c9584286811e47571/pyarrow-0.6.0-cp35-cp35m-macosx_10_6_intel.whl#md5=e186c34dad94bfd147812fa29cf1e487 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/85/ac/f49561639121a8e8e6d3dee63bdf8d6f4c1a1885fd47ffe4a65c2ecf274b/pyarrow-0.6.0-cp35-cp35m-win_amd64.whl#md5=ecb8d8fc59008fa7c740e7b670019212 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/86/81/ef15c84686352d26f5fcececd136c48fa41652190d57b973dd807edf8bad/pyarrow-0.5.0.post2-cp35-cp35m-manylinux1_x86_64.whl#md5=013d635c0a827d09407a02f39496ce6e (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/8b/d9/9b1a8aedb2059e5386d218a60f9e52523b8983fd4164e3f2abe68e4a79d7/pyarrow-0.7.0-cp36-cp36m-manylinux1_x86_64.whl#md5=10482563d47380cfd287499dd9227242 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/8c/5a/95c6229bd8551d95e29438fb94f12223343b08b219ad70ba98a39cde60f2/pyarrow-0.7.0-cp27-cp27m-macosx_10_6_intel.whl#md5=8fd7dd7cf0a8bc90c5ff636afc366d9f (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/8f/ad/477ad4e28654e2bcdc2fb082fb557d08c2b9682b5056e6358a5fae9666c6/pyarrow-0.8.0-cp27-cp27mu-manylinux1_x86_64.whl#md5=0f744dea1534fdb3de10cc05419c0bae (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/90/96/8901abac3e277f53192a08ce25ea68f234c5caef21a80416bbd3f64af850/pyarrow-0.7.0-cp27-cp27mu-manylinux1_x86_64.whl#md5=a9efb0eb1aafd7a91ba20eae7aad8105 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/92/cb/72cc51038ab648da0989850179177407ade4e02ade7c2434869215233f3a/pyarrow-0.3.0-cp35-cp35m-manylinux1_x86_64.whl#md5=4c1ff8859cfcb750dc5288bf6a18986b (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/9e/13/cd52edaec2952b4e2d522adbc1bdc6785a38694d0f180027d1b9059b1d23/pyarrow-0.2.0-cp27-cp27mu-manylinux1_x86_64.whl#md5=b7ab1c2741e9d16778d13a4244c316c7 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/9f/be/112e9957c59654328dfff0cc22d88a4c57f80a942f1dcec4041ce9fdb75e/pyarrow-0.4.0-cp35-cp35m-manylinux1_x86_64.whl#md5=747f950b72348c7d6d464764e0a4e530 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/a3/f8/b38b4ebf99671366b44d118e67fd368918f5beca1b244e0b9b59236a6475/pyarrow-0.6.0-cp35-cp35m-manylinux1_x86_64.whl#md5=8ea5ed8f1f216a7d8c7485284a624dd7 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/a5/3e/a673de0dbc953b36d88129c4fdac4c2d1f375833a6511ce070e4e15df93a/pyarrow-0.3.0-cp27-cp27mu-manylinux1_x86_64.whl#md5=5115fbad5bc85817339e1715210ab231 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/ab/60/d06ece4610a7b71fa8ffa44b1f31d13098658a3d436c6d5a985dd0d9a602/pyarrow-0.2.0-cp36-cp36m-manylinux1_x86_64.whl#md5=7f1bb5cbd6f3b4e4723fd4fdf8df5407 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/ad/36/a082258bb6cda3c32deed657f8b76619f06f191e7e968ba36ed335c85c9e/pyarrow-0.4.1-cp27-cp27m-macosx_10_6_intel.whl#md5=1f522853c5ef050a409bd552dcd40038 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/ae/93/66641452f6f20a4109dc5957bd3efd7325d041eabeb338438a07e6ec5cc9/pyarrow-0.5.0-cp35-cp35m-macosx_10_6_intel.whl#md5=00298733c4f607302dc378c6904f78f5 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/b4/3c/3b8ae2d3ec9bb64c2d5000194c81274cb5bf216d3bb24c9bccc8e589a67a/pyarrow-0.7.1-cp36-cp36m-macosx_10_6_intel.whl#md5=d7ec9f93854ba5c4520b3a81a613ecb1 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/b6/a0/13ea3b6b041ad67f836fbb94ff7a5b54b40b5a15b93cda672e1369c27395/pyarrow-0.4.1-cp35-cp35m-win_amd64.whl#md5=cac183724cb86808129735afea0e9184 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/b8/2c/27ecf27f929d1793db3333c0e381f7b43e3d182eb7b5104323889a661091/pyarrow-0.7.1-cp35-cp35m-manylinux1_x86_64.whl#md5=ff304289e5ee1ef9ccd5c641cf3ddb4a (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/bd/32/dcdac5edd3216c9363abec37f74f7c905518c1a5172c8a5085ef872b1122/pyarrow-0.7.0-cp35-cp35m-win_amd64.whl#md5=baa0e569753adda7f247f9fc7c811917 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/bd/34/f9b756ba88266405f34db3fd6163ef78a369cc9ecb507a04358968d65129/pyarrow-0.4.1-cp36-cp36m-manylinux1_x86_64.whl#md5=ae97b804ac5e0b6fc3478cc84f2e26e4 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/c1/fb/b26445d66c81167ec5ff64ed18a82a670afdc5c542fa3f1f94e6222dd1ee/pyarrow-0.4.1-cp36-cp36m-win_amd64.whl#md5=2b40c5e72f55afa6e1d3f573d4bd3780 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/c3/00/4f8e1b9eae8600302cd448aa11ba4de990e9245159fd956ebe4e3a0b85c8/pyarrow-0.4.0-cp35-cp35m-macosx_10_6_intel.whl#md5=8efbb15a7c834608b1278bd014c14a74 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/c4/86/6d38bc9f5d52c5aa601efa056b0f827994d5c60090bdd7aab49338e87a6f/pyarrow-0.5.0.post2-cp36-cp36m-manylinux1_x86_64.whl#md5=c1c30f78a9de9558efc9f5fb761efd39 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/c5/67/dcb88001853ffc7df31bb68ff5315d6ce86fd169a214daec770512176388/pyarrow-0.7.0-cp35-cp35m-macosx_10_6_intel.whl#md5=a306db46e9b64af40f49ac868be1eb7d (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/c6/f6/c7d2dca9560d494e236b86d631ffee1e2da78b4a49ee84300242564136a9/pyarrow-0.3.0-cp34-cp34m-manylinux1_x86_64.whl#md5=d75688f5a60c504c82d2f4dead76b67d (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/cb/18/6237368e3cba9aa347d5a8163dc0d3b4e8c4b7edbe3fce6b876f421bc333/pyarrow-0.4.1-cp35-cp35m-manylinux1_x86_64.whl#md5=3c4452cfed0105879963925503375443 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/d6/3b/a83433c53a7c9dd2bb5e3eefa6a086c0aae058128138e7b42a6897e657b6/pyarrow-0.4.0-cp27-cp27mu-manylinux1_x86_64.whl#md5=64bf9e61538faacd25b89ea1baab3b29 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/de/53/5852cbe209c593248eb301832dbbf1e770ebca240d859cefbb45b2515f72/pyarrow-0.6.0-cp36-cp36m-macosx_10_6_intel.whl#md5=1b7ce613cbfc80a6b7a41100768bf9d4 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/e1/d6/5c62fbab9c8127dedc0297e1307f8b16f2a9af7749e11c5eb3635e2400e2/pyarrow-0.2.0-cp34-cp34m-manylinux1_x86_64.whl#md5=fcc241638cd9f83a1f7f81f4740e6ea2 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/e6/8c/ecad1138c380512aa5787aa289fe543163639f835937b439a2887ceba7eb/pyarrow-0.8.0-cp35-cp35m-macosx_10_6_intel.whl#md5=05f98ccb720fb98a41be14c1fe199a86 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/e6/f4/5e465e430e04e68355f4ec6e63a61dc64085997b4d4154c154c0d723af95/pyarrow-0.5.0.post2-cp27-cp27mu-manylinux1_x86_64.whl#md5=880f7b6a2dcb9bdf175728ceac49939f (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/ed/ae/3febb068ea9817dfdeac03adcd6cd6ed4663dbe47d789aab89571b23b60e/pyarrow-0.7.0-cp35-cp35m-manylinux1_x86_64.whl#md5=cf294476195e8de038151981454d4040 (from https://pypi.python.org/simple/pyarrow/); it is not compatible with this Python
  Could not find a version that satisfies the requirement pyarrow (from versions: )
Cleaning up...
No matching distribution found for pyarrow
Exception information:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/local/lib/python3.6/site-packages/pip/commands/install.py", line 335, in run
    wb.build(autobuilding=True)
  File "/usr/local/lib/python3.6/site-packages/pip/wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "/usr/local/lib/python3.6/site-packages/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/usr/local/lib/python3.6/site-packages/pip/req/req_set.py", line 554, in _prepare_file
    require_hashes
  File "/usr/local/lib/python3.6/site-packages/pip/req/req_install.py", line 278, in populate_link
    self.link = finder.find_requirement(self, upgrade)
  File "/usr/local/lib/python3.6/site-packages/pip/index.py", line 514, in find_requirement
    'No matching distribution found for %s' % req
pip.exceptions.DistributionNotFound: No matching distribution found for pyarrow

Notice the 'Skipping link ... it is not compatible with this Python` messages. They happen because the wheel format is currently incompatible with musl libc.

This means we must publish the source distribution to allow pip to fallback to compilation of arrow.

 

Reporter: Omer Katz / @thedrow

Related issues:

Note: This issue was originally created as ARROW-2058. Please see the migration documentation for further details.

@asfimport
Copy link
Collaborator Author

Wes McKinney / @wesm:
Enabling a source build with pip install pyarrow seems like a pretty large undertaking, but I won't dissuade you from trying!

@asfimport
Copy link
Collaborator Author

Wes McKinney / @wesm:
Do many projects provide alpine wheels? What is their build toolchain like?

@asfimport
Copy link
Collaborator Author

Uwe Korn / @xhochy:
You cannot provide these wheels on PyPI. There is no platform tag yet that indicates the use of musl-libc. The musl/Alpine community must first invest in a standard like manylinux1 (which is only for glibc based distros).

If there was an alternative Package repository for wheels on Alpine Linux, we could upload wheels there but this also does not seem to exist.

Closing as "Won't fix" until the Alpine community has adressed this.

@asfimport
Copy link
Collaborator Author

Adam Davis:
Created on 17-Mar-2021, PEP 656 establishes the musllinux standard for binary Python package distributions for musl-based Linux distributions such as Alpine. Would it be possible to reopen this issue?

@MrMino
Copy link

MrMino commented Aug 2, 2023

Looks like this could be reopened now. This is a bit of a pressing issue given that pandas is depending on pyarrow, and maintainers expressed interest in releasing wheels for Alpine.

@kou
Copy link
Member

kou commented Aug 2, 2023

OK. I reopen this.
Does someone want to work on this?

@kou kou reopened this Aug 2, 2023
@mbacarella
Copy link

mbacarella commented Feb 15, 2024

We need this too. I can maybe take a crack at it though I'm pretty unfamiliar with Python packaging and wheels.

Is it basically copy/paste this file https://github.com/apache/arrow/blob/main/ci/docker/python-wheel-manylinux.dockerfile and hack stuff so that it's Alpine instead of CentOS?

@kou
Copy link
Member

kou commented Feb 15, 2024

Does PyPA provide a Docker image for musllinux wheel like https://quay.io/repository/pypa/manylinux2014_x86_64?tab=tags ?

@MrMino
Copy link

MrMino commented Feb 15, 2024

@kou yes, several, just filter by musl on this page: https://quay.io/organization/pypa

@kou
Copy link
Member

kou commented Feb 15, 2024

OK. Let's use these images as base images.

@mbacarella
Copy link

Anything I can do to help? Happy to test and provide feedback. Or donate to individuals or foundations.

@kvanzuijlen
Copy link

@kou I'm willing to work on this, although I'm not familiar with the code base. Can you give me some pointers to what needs to be done?

@kou
Copy link
Member

kou commented Feb 21, 2024

Sure. Here are related things:

  1. docker-compose.yml: https://github.com/apache/arrow/blob/main/docker-compose.yml
    • We need to add python-wheel-musllinux-* like existing python-wheel-manylinux-*
  2. ci/docker/: https://github.com/apache/arrow/blob/main/docker-compose.yml
    • We need to add python-wheel-mulslinux*.dockerfile like python-wheel-manylinux*.dockerfile for the above docker-compose.yml change
  3. ci/scripts/: https://github.com/apache/arrow/tree/main/ci/scripts
    • We need to add python_wheel_musllinux_build.sh like existing python_wheel_manylinux_build.sh
    • We may be able to create an unified build script like python_wheel_linux_build.sh when we can unify python_wheel_manylinux_build.sh and python_wheel_musllinux_build.sh
  4. dev/tasks/tasks.yml: https://github.com/apache/arrow/blob/main/dev/tasks/tasks.yml
    • We need to add wheel-musllinux* like existing wheel-manylinux-* to run python-musllinux-* services that are added by 1. in our nightly jobs

(We may need to change another files too...)

@numbworks

This comment was marked as duplicate.

@nveloso
Copy link

nveloso commented Feb 2, 2025

Hello guys!
I'm trying to add python wheels for musl linux (alpine) but I can't test it due to how it installs vcpkg.
My branch is in my fork here.
Can you help me on how I can install vcpkg without needing the GITHUB_* environment variables?
Thank you.

@kou
Copy link
Member

kou commented Feb 2, 2025

How did you test it on your local?
Did archery docker run python-wheel-musllinux-1.2 report an error?

@nveloso
Copy link

nveloso commented Feb 3, 2025

It's during the build process when calling install_vcpkg.sh.
I tried to comment out the export GITHUB_* variables but the script needs them.
I run it with docker-compose build python-wheel-musllinux-1-2.

@kou
Copy link
Member

kou commented Feb 4, 2025

@nveloso
Copy link

nveloso commented Feb 6, 2025

Hey, sorry for the delay. I was trying to reproduce the bug but I couldn't try.
Actually I was able to generate a wheel for alpine 🙃
I had to change a few things in order to succeed on my ARM mac. It will be difficult for me to test on an x86_64 architecture. I can try during the weekend but I'm not sure if will make it.

I had to disable the VCPKG_DEFAULT_TRIPLET environment variable in the dockerfile and in the build script because the script to install vcpkg was trying to install a triplet for a different CPU architecture. I tried setting to some osx and aarch64 but none succeed. I let it find the best triplet for me by disabling that environment variable.
When it was trying to build it failed on boost library. I figured it was related with the triplet change. The triplet installed things on /opt/vcpkg/installed/arm64-linux and everything else expects it to be installed on /opt/vcpkg/installed/. What I did was to copy everything from /opt/vcpkg/installed/arm64-linux to /opt/vcpkg/installed/. After this I was able to generate a wheel.

I would like to test building a wheel on a x86_64 architecture. I think in that architecture I don't need to disable the VCPKG_DEFAULT_TRIPLET environment variable and will make things smoother. I think the github runners also run on that architecture right?

@kou
Copy link
Member

kou commented Feb 7, 2025

You don't need to change VCPKG_DEFAULT_TRIPLET in Dockefile. You can change it by environment variables something like: ARCH=arm64v8 ARCH_SHORT=arm64 archery docker run python-wheel-musllinux-1-2

@nveloso
Copy link

nveloso commented Feb 8, 2025

That worked perfectly. I was setting it to a wrong value. I was able to build a wheel.
I'm working on adding the tests but they are failing. I'm running them through the python_wheel_unix_test.sh script. I have 2 tests failing:

  • test_uwsgi_integration
  • test_print_stats

I believe the root cause is the same which is related to this:
/arrow/cpp/src/arrow/filesystem/s3fs.cc:3461: arrow::fs::FinalizeS3 was not called even though S3 was initialized. This could lead to a segmentation fault at exit !!! uWSGI process 3487 got Segmentation Fault !!!

Do you have any idea of what it might be?

Here are some logs of the failed tests:

====================================================================================== FAILURES ======================================================================================
_______________________________________________________________________________ test_uwsgi_integration _______________________________________________________________________________

    @pytest.mark.s3
    def test_uwsgi_integration():
        # GH-44071: using S3FileSystem under uwsgi shouldn't lead to a crash at shutdown
        try:
            subprocess.check_call(["uwsgi", "--version"])
        except FileNotFoundError:
            pytest.skip("uwsgi not installed on this Python")

        port = find_free_port()
        args = ["uwsgi", "-i", "--http", f"127.0.0.1:{port}",
                "--wsgi-file", os.path.join(here, "wsgi_examples.py")]
        proc = subprocess.Popen(args, stdin=subprocess.DEVNULL)
        # Try to fetch URL, it should return 200 Ok...
        try:
            url = f"http://127.0.0.1:{port}/s3/"
            start_time = time.time()
            error = None
            while time.time() < start_time + 5:
                try:
                    with urlopen(url) as resp:
                        assert resp.status == 200
                    break
                except OSError as e:
                    error = e
                    time.sleep(0.1)
            else:
                pytest.fail(f"Could not fetch {url!r}: {error}")
        finally:
            proc.terminate()
        # ... and uwsgi should gracefully shutdown after it's been asked above
>       assert proc.wait() == 30  # UWSGI_END_CODE = 30
E       AssertionError: assert -11 == 30
E        +  where -11 = wait()
E        +    where wait = <Popen: returncode: -11 args: ['uwsgi', '-i', '--http', '127.0.0.1:49245', '...>.wait

usr/local/lib/python3.9/site-packages/pyarrow/tests/test_fs.py:2052: AssertionError
-------------------------------------------------------------------------------- Captured stdout call --------------------------------------------------------------------------------
2.0.28
-------------------------------------------------------------------------------- Captured stderr call --------------------------------------------------------------------------------
*** Starting uWSGI 2.0.28 (64bit) on [Sat Feb  8 18:56:14 2025] ***
compiled with version: 13.2.1 20231014 on 31 October 2024 19:02:44
os: Linux-6.8.0-50-generic #51-Ubuntu SMP PREEMPT_DYNAMIC Sat Nov  9 18:03:35 UTC 2024
nodename: ae5a02215122
machine: aarch64
clock source: unix
pcre jit disabled
detected number of CPU cores: 4
current working directory: /
detected binary path: /usr/local/bin/python3.9
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
*** WARNING: you are running uWSGI without its master process manager ***
your memory page size is 4096 bytes
detected max file descriptor number: 1048576
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uWSGI http bound on 127.0.0.1:49245 fd 4
spawned uWSGI http 1 (pid: 3488)
uwsgi socket 0 bound to TCP address 127.0.0.1:40033 (port auto-assigned) fd 3
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
Python version: 3.9.19 (main, Mar 20 2024, 20:45:15)  [GCC 12.2.1 20220924]
--- Python VM already initialized ---
Python main interpreter initialized at 0xeff90822b840
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 72904 bytes (71 KB) for 1 cores
*** Operational MODE: single process ***
WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0xeff90822b840 pid: 3487 (default app)
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
spawned uWSGI worker 1 (and the only) (pid: 3487, cores: 1)
[pid: 3487|app: 0|req: 1/1] 127.0.0.1 () {30 vars in 346 bytes} [Sat Feb  8 18:56:14 2025] GET /s3/ => generated 12 bytes in 20 msecs (HTTP/1.1 200) 1 headers in 44 bytes (1 switches on core 0)
/arrow/cpp/src/arrow/filesystem/s3fs.cc:3461:  arrow::fs::FinalizeS3 was not called even though S3 was initialized.  This could lead to a segmentation fault at exit
!!! uWSGI process 3487 got Segmentation Fault !!!
________________________________________________________________________ test_print_stats[system_memory_pool] ________________________________________________________________________

pool_factory = <cyfunction system_memory_pool at 0xe04d9b5c3ad0>

    @pytest.mark.parametrize('pool_factory', supported_factories())
    def test_print_stats(pool_factory):
        code = f"""if 1:
            import pyarrow as pa

            pool = pa.{pool_factory.__name__}()
            buf = pa.allocate_buffer(64, memory_pool=pool)
            pool.print_stats()
            """
        res = subprocess.run([sys.executable, "-c", code], check=True,
                             universal_newlines=True, stdout=subprocess.PIPE,
                             stderr=subprocess.PIPE)
        if sys.platform == "linux":
            # On Linux at least, all memory pools should emit statistics
>           assert res.stderr.strip() != ""
E           AssertionError: assert '' != ''
E            +  where '' = <built-in method strip of str object at 0xe04d9c3ec6f0>()
E            +    where <built-in method strip of str object at 0xe04d9c3ec6f0> = ''.strip
E            +      where '' = CompletedProcess(args=['/usr/local/bin/python', '-c', 'if 1:\n        import pyarrow as pa\n\n        pool = pa.system...= pa.allocate_buffer(64, memory_pool=pool)\n        pool.print_stats()\n        '], returncode=0, stdout='', stderr='').stderr

usr/local/lib/python3.9/site-packages/pyarrow/tests/test_memory.py:295: AssertionError

There is also a lot of skipped tests (603) and I'm not sure if this is ok. Here is the final report:
============================================= 2 failed, 7200 passed, 603 skipped, 12 xfailed, 2 xpassed, 5 warnings in 80.21s (0:01:20) ==============================================

@kou
Copy link
Member

kou commented Feb 9, 2025

Could you open a PR? Let's discuss details at there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants