Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Python implementation with native Rust module #547

Merged
merged 72 commits into from
Oct 30, 2024
Merged

Conversation

Esgrove
Copy link
Contributor

@Esgrove Esgrove commented Oct 25, 2024

Adds a separate Python vault version under python-pyo3 that uses the Rust vault internally.

pyo3 is used to create Python bindings for the Rust vault. There is a single lib.rs file that wraps the Rust vault library. It simply has one function for each CLI command that takes parameters from python side and calls the matching rust cli command. Pyo3 has an additional tool maturin which is used to init the project, and handle building and publishing.

The Python side only has the CLI specification that was written from scratch to match the Rust CLI (uses subcommands so all instead of --all). All actual functionality happens in the rust module (including print output for the most part). To enable this, I moved the Rust vault cli.rs to be part of the Rust lib, so it can be reused for python. Otherwise would need to duplicate all that in Python, going against one of the primary goals for this whole thing. This does lead to a bit of code duplication for passing around the arguments since each command creates its own vault instance but this way the communication between rust and python is very simple and kept to a minimun.

There is an experimental-async feature for pyo3 that would enable async support between rust and python, but since we don't actually need to do anything async directly in the CLI, I did this the easy way and just wrapped all rust code inside a separate runtime invocation for each function with a blocking task, so the python side does not need to care about async at all.

Typer does not seem to support giving commands flag aliases like we have in the rust version. Currently it does not have the argument flags at all, so it is not backwards compatible with the old python cli interface.

  • Add new python vault cli using the rust lib
  • Update GitHub actions, run python checks for new cli and include it in integration tests
  • Move Rust vault cli module to library
  • Add wheel build job for rust python package using maturin generated gha workflow

@Esgrove Esgrove marked this pull request as ready for review October 28, 2024 11:07
@Esgrove Esgrove requested a review from kahlstrm October 28, 2024 11:07
Copy link
Contributor

@kahlstrm kahlstrm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, haven't tested this on my own but if it works then 🤷

One thing that could clean things up would be to separate the CLI and the library code to separate crates (using cargo workspace perhaps?) so that the library wouldn't download and compile all the CLI dependencies, thus reducing binary size.

rust/src/lib.rs Outdated Show resolved Hide resolved
@Esgrove Esgrove merged commit 756a23b into master Oct 30, 2024
19 checks passed
@Esgrove Esgrove deleted the rust-python-module branch October 30, 2024 13:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants