Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cross-platform trajectory output #107

Open
jmineau opened this issue Sep 17, 2024 · 3 comments · May be fixed by #108
Open

cross-platform trajectory output #107

jmineau opened this issue Sep 17, 2024 · 3 comments · May be fixed by #108

Comments

@jmineau
Copy link
Member

jmineau commented Sep 17, 2024 •
edited
Loading

While R is great, and the bindings that @benfasoli wrote for STILT are incredible, for users, like myself, whose language of choice is something other than R, outputting the trajectories to rds format is quite inconvenient. Sure, we could use R conversion libraries like rpy2, but it would be much more convenient if the trajectories could be written to a cross-platform, portable format.

Some ideas include:

  • HDF5 - offers compression to columnar data with the added support of attributes
  • netCDF4 - standard in geoscience, less compressed?
@jmineau jmineau linked a pull request Sep 17, 2024 that will close this issue
Open
@jmineau
Copy link
Member Author

jmineau commented Sep 17, 2024

I implemented the option to write to h5 in my PR: #108

@benfasoli
Copy link
Contributor

HDF5 and NetCDF4 are probably already installed since the footprints are NetCDF4 but they aren't very ergonomic for users and usually have clunky install stories because the language bindings are links to the system-level HDF5/NetCDF4 C-lib installs.

Take a look at parquet files for a performant column storage format that works cross-platform without many of the same drawbacks 👍

@jmineau
Copy link
Member Author

jmineau commented Sep 20, 2024

Appreciate the insight.

I have implemented parquet as an option and added documentation. I left h5 as an option, but didnt document in the markdown docs. I also added the option and documentation to disable writing trajectories with ''

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

implement cross-platform trajectory output jmineau/stilt
2 participants
@benfasoli @jmineau