Fix WKB writing on big-endian systems

[QuLogic]

shapely.to_wkb (called in pyogrio.geopandas.write_dataframe via geopandas.array.to_wkb) defaults to machine byte order, so simply reading the first byte (of a 4-byte integer geometry type) is broken on big-endian machines. And since some geometry type could be 2 bytes, this also seems broken on little-endian machines, though I did not check if any 2-byte types could be produced.

The commented-out code here indicates the correct course of action (reading the first byte to determine byte order).

This fixes the 190 tests that go through this function and fail on big-endian.

[QuLogic]

Note, if you'd prefer to not use the Python int.from_bytes, I could write it with bytes and bitshifting. This usually compiles to a single instruction with optimizing compilers (though I'm unsure about the result through Cython.)

[QuLogic]

Note, if you'd prefer to not use the Python int.from_bytes, I could write it with bytes and bitshifting. This usually compiles to a single instruction with optimizing compilers (though I'm unsure about the result through Cython.)

Since Windows didn't like the int.from_bytes, I decided to implement this. (Probably it just needed the explicit type, but I can't check here.) I also changed wkb_buffer to const as it doesn't need to be written to, and potentially saves a copy using it directly.

[brendan-ward]

Thanks @QuLogic ! Can you please add an entry to the changelog?

Do you have suggestions on how we can test for this in CI? I'm not seeing a straightforward way to do this with Github Actions, and it seems like it would require building the full stack GDAL, GEOS, Shapely, etc for something like s390x.

[QuLogic]

Do you have suggestions on how we can test for this in CI? I'm not seeing a straightforward way to do this with Github Actions, and it seems like it would require building the full stack GDAL, GEOS, Shapely, etc for something like s390x.

There's definitely no native support, but you could run it in a container (as was done for ARM previously). I would use system packages and you would get all of them pre-compiled (at the expense of not always being the latest). I'm sure Debian exists for s390x, or Fedora as I'm using.

[rouault]

Do you have suggestions on how we can test for this in CI?

cf OSGeo/gdal#11375 for an example how to do that for C/C++ software

[brendan-ward]

@QuLogic would you mind please merging in the latest from main? We recently fixed the outstanding CI issues. Once everything shows up green here, I think we can go ahead and merge this, and save implementing specific CI tests for big endian systems for another PR.

[QuLogic]

Looks good now.

[brendan-ward]

Thanks @QuLogic !