You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
For the Spark interface it would be nice to implement a proxy for accessing remote ironArray arrays so that we don't need a shared filesystem.
This can be achieved quite easily with a similar interface than the current proxy for Zarr objects. We just need to implement a network interface to our ironArray objects and ask for metainfo (shape, dtype...) and slices remotely.
As a proof of concept, we can use FastAPI (for the server) and requests (for the client).
The text was updated successfully, but these errors were encountered:
This work is being done in inaos/iron-array-python#189. This work is very preliminary and it doesn't let compute a reduction of a remote viewed array, because it doesn't copy the array before computing the reduction.
For the Spark interface it would be nice to implement a proxy for accessing remote ironArray arrays so that we don't need a shared filesystem.
This can be achieved quite easily with a similar interface than the current proxy for Zarr objects. We just need to implement a network interface to our ironArray objects and ask for metainfo (shape, dtype...) and slices remotely.
As a proof of concept, we can use FastAPI (for the server) and requests (for the client).
The text was updated successfully, but these errors were encountered: