up4: Universal Post-Processor for Particulate Processes

A fast and flexible analysis toolset for particle data of all kinds

Use it to compare inter-platform particle results - either experimental or synthetic data generated by different platforms like LIGGGHTS, MercuryDPM, Barracuda and more.

up4 has a python API, which enables it to be easy to use for a wide range of scientists, but the core of it is implemented in Rust.

Installation

1. HDF5

One of the main requirements is the HDF5 library. To install the library we recommend using conda

conda install -c conda-forge hdf5==1.10.4

additionally, we recommend installing to export the following environment variables:

export HDF5_DIR="$CONDA_PREFIX"

for more information see the hdf5-rust instructions

2. Rust

you will need a rust compiler, see:

https://www.rust-lang.org/tools/install

Rust dependencies will be installed automatically DURING compilation

3. Python

Download or clone this repository and run:

python3 -m pip install -r requirements.txt

after the requirements are installed, you can install the package with:

python3 -m pip install .

this could take some time as it compiles the rust code on your machine.

To ensure that the installation was successful, you can run tests with pytest:

python3 -m pytest tests/test_all.py

VSCode users

Currently, type hinting/ autocomplete is only possible if the Python language server is set to Jedi.

To DO / Dev notes

Tests

• test failures. make exceptions work
• make paths work also when pytested from different folder
• new, better and smaller test datasets
• test converter::convertertools:{interpolate, velocity}

Ideas

• save_constraints --> save the current particle selector constraints in the hdf5 file if file contains constraints, load them in Selector, however warn the user while loading, also make a reverse function to set selector to default state ( data.selector_default or similar)
• Align grids for comparison. Use some smart metric to align two grids and make diffs/or Hanquiao plots
• Statistics, is my data good enough? (how many traj do i need per cell to get good data)
• Data manipulation --> Rotate, translate system(with backup)

Bugs

• bug in interpolator for csv files? there are zeros in 3u_HD1_glass

Documentation

Extensive effort has gone into documenting every trait, function, struct, class and method in this library, so regardless of using this library in Rust or Python, there are no magic undocumented blocks of code. The Rust API reference can be found here

TODO add link

and the Python API reference here.

Help and Support

We recommend you check out our tutorials. If your issue is not suitably resolved there, please check the issues page on our GitHub. Finally, if no solution is available there, feel free to open an issue; the authors will attempt to respond as soon as possible.

Performance

The core of this library is implemented in Rust, a modern systems language that natively boasts high performance and strong memory & thread safety. The pyo3 Rust library is used to export the Rust backend as a Python library, so Rust knowledge is not a prerequisite for using up4. We deliberately chose a Rust-Python mix to ensure that the computationally intensive parts are fast but readable (rather than using an arcane mix of Cython and Numba) whilst maintaining an easy to use interface, which is where Python excels.

Contributing

The up4 library is not a one-man project; it is continually being worked on directly and indirectly by a dedicated group of granular materials researchers who are both tired of writing the same Python boilerplate to analyse experiments and waiting for said code to execute. If you are reading this, then it is likely that you feel at least one of these frustrations too!

If you want to contribute to this project, there are many ways to help:

• Open an issue
• Write a tutorial that fills in any gaps in the up4 documentation to help future users.
• Share relevant algorithms/ functions that improve the post-processing process, so that others can benefit. If these come from published work, then please add a reference to this work in the code documentation.
• Add your code directly to the repo and open a pull request.

Citing

If you use this library in your research, you are kindly asked to cite:

[Paper after publication]

Licensing

The up4 package is GPL v3.0 licensed. In non-lawyer terms, the key points of this license are:

• You can view, use, copy and modify this code freely.
• Your modifications must also be licensed with GPL v3.0 or later.
• If you share your modifications with someone, you have to include the source code as well.

Essentially, do whatever you want with the code, but don't try selling it saying it's yours :). GPL v3.0 is indeed a very strong copyleft license; it was deliberately chosen to maintain the openness and transparency of great software and progress. This also ties into the very reason for up4's conception - to transparently process and compare Lagrangian data, regardless of source. Further, open collaboration is frankly way more efficient than closed, for-profit competition.

PoincarePlotter

Project to-do list

• HDF5 converter
  • VTK -> HDF5
  • CSV -> HDF5
  • TXT -> HDF5
  • XLSX -> HDF5
• Plotting utilities
  • Vector plots
    • 2D
    • 3D
  • Heatmaps
  • Contour plots
  • Slice plots
  • Distributions
  • Parity plots
  • Recipes for above plots
• Vectorfield functions
  • Grid
    • Euler grid
    • Lagrange grid
• Equipment specific functions (or recipes?)
  • Rotating drum
  • Mills
  • Fluidised beds
• Generic particle functions (see old upppp)
  • Granular temperature distribution
  • Velocity distribution
• Documentation
  • Agree standards
  • Python
    • Configure Sphinx
    • Link rust docs (if possible)
  • Rust
• DEM contact list functions
  • Energy dissipated
  • Collision modes

State of up4

Plotting

• Rust
  • Limited ability to easily customise plots compared to the Python API. This is actively being worked on.
• Python
  • Export of created JSON strings to Plotly.py means that all tools are available to edit plot layout.