Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rust prost tonic #265

Merged
merged 31 commits into from
Sep 8, 2023
Merged

Conversation

echochamber
Copy link
Contributor

@echochamber echochamber commented Jun 6, 2023

Status: It works!


Updated version of @titanous PR #202.

Users will need to manually declare the declared_proto_packages=["proto.package.name", ...]. The compile rule also requires the crate_name=my_crate be provided (but the macros take care of this part for you).

However, with this information the custom rule implementation will handle creating all the necessary --externs_path=.proto.package.name=::my_create::proto::package::name mappings for you (prost docs).

Things I've tested on both rust_prost_proto_library and rust_tonic_grpc_library:

  • nested.proto.namespaces
  • A single compile action with more than one declared proto package.
  • Compiling a proto which imports a type from another proto package + depending on its respective rust_prost_proto_library.
  • Serde plugin.
  • Crate plugin.
  • Using WKT.
  • PBJson

Known limitations:

  • Compilation fails whe trying to deriving traits for types that contain a pbjson type which doesn't already implement that trait. (titanous has a custom plugin based solution here that I haven't dug into yet).

@echochamber echochamber marked this pull request as draft June 6, 2023 01:40
@github-actions github-actions bot added dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation lang-rust Rust rules specific labels Jun 6, 2023
@echochamber
Copy link
Contributor Author

echochamber commented Jun 6, 2023

@aaliddell The majority of the logic here is done and ready for review/initial feedback. All of the targets in rust/example/ should build and run just fine.

I'm still working on getting the rules to work outside of the main rules_proto_grpc Workspace. Currently getting an error when I try to bazel build //:thing_rust_proto in the example/rust/rust_prost_proto_compile/WORKSPACE.

external/rules_proto_grpc/rust/3rdparty/crates/BUILD.bazel:161:6: no such package '@rules_proto_grpc_rust__protoc-gen-prost-serde-0.2.3//': The repository '@rules_proto_grpc_rust__protoc-gen-prost-serde-0.2.3' could not be resolved: Repository '@rules_proto_grpc_rust__protoc-gen-prost-serde-0.2.3' is not defined and referenced by '@rules_proto_grpc//rust/3rdparty/crates:protoc-gen-prost-serde__protoc-gen-prost-serde'

@echochamber
Copy link
Contributor Author

And it works! All of the examples build and run smoothly.

That said, there is something interesting going on with deriving the hash/eq type for the pbjson types.

  1. Thing currently cannot have hash derived because ::pbjson_types::Any doesn't implement Hash.
  2. Attempting to derive Hash on Place (which depends upon thing) causes Prost to also attempt to derive Hash for Thing (since one of the fields in Place is an instance of Option.
  3. This causes a build failure.

However this only occurs in this second example code snippet below, in the first the build succeeds (even though Thing doesn't derive Hash in the generated code), which is interesting.

This builds
rust_tonic_grpc_library(
    name = "greeter_tonic",
    declared_proto_packages = ["example.proto"],
    protos = [
        "@rules_proto_grpc//example/proto:greeter_grpc",
        "@rules_proto_grpc//example/proto:person_proto",
        "@rules_proto_grpc//example/proto:place_proto",
    ],
    options = {
        "//rust:rust_prost_plugin": [
            "type_attribute=.example.proto.Person=#[derive(Eq\\,Hash)]",
            "type_attribute=.example.proto.Place=#[derive(Eq\\,Hash)]",
    ]},
    prost_proto_deps = [
        ":thing_prost"
    ]
)
rust_prost_proto_library(
    name = "thing_prost",
    declared_proto_packages = ["example.proto"],
    protos = [
        "@rules_proto_grpc//example/proto:thing_proto",
    ],
)
This doesn't build
rust_tonic_grpc_library(
    name = "greeter_tonic",
    declared_proto_packages = ["example.proto"],
    protos = [
        "@rules_proto_grpc//example/proto:greeter_grpc",
        "@rules_proto_grpc//example/proto:person_proto",
        "@rules_proto_grpc//example/proto:place_proto",
        "@rules_proto_grpc//example/proto:thing_proto",
    ],
    options = {
        "//rust:rust_prost_plugin": [
            "type_attribute=.example.proto.Person=#[derive(Eq\\,Hash)]",
            "type_attribute=.example.proto.Place=#[derive(Eq\\,Hash)]",
    ]},
)
build error
error[E0277]: the trait bound `Thing: Hash` is not satisfied
  --> bazel-out/k8-fastbuild/bin/greeter_tonic_pb_fixed/example.proto.rs:17:5
   |
10 | #[derive(Eq, Hash)]
   |              ---- in this derive macro expansion
...
17 |     pub thing: ::core::option::Option<Thing>,
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `Hash` is not implemented for `Thing`
   |
   = help: the trait `Hash` is implemented for `std::option::Option<T>`
   = note: required for `std::option::Option<Thing>` to implement `Hash`
   = note: this error originates in the derive macro `Hash` (in Nightly builds, run with -Z macro-backtrace for more info)

@echochamber echochamber changed the title [WIP] Rust prost tonic Rust prost tonic Jun 6, 2023
@echochamber echochamber marked this pull request as ready for review June 6, 2023 05:16
titanous and others added 3 commits June 5, 2023 22:28
Currently we default to rules_proto version which, which is out of date
and missing @platforms//os:fuchsia. This causes any downstream rulesets
that are configuring using this constraint (rules_rust) to fail to
build.
@aaliddell
Copy link
Member

Ok, time for me to get reading. Thankfully most of those 20,000 lines changed are just raze -> crate_universe 😱

I'm trying to think if there's any tricks we can play with an aspect that'd let us discover the externs automatically

@aaliddell
Copy link
Member

As an update of where I am with this:

  • Overall 👍
  • Some reorganisation is needed to standardise to the rulegen layout so that these rules aren't a special case to maintain, but I can do that since it's a pain to do
  • On more thinking, I'm more convinced an aspect may be able to help with the naming of the externs, but I won't know for sure until I try it out

I'll start adding commits here as I do the above

@github-actions github-actions bot added the lang-cpp C++ rules specific label Sep 8, 2023
@aaliddell
Copy link
Member

I've spun the aspect investigation off into #282 as future work, so it doesn't hold this up any further. The rest is now passing with rulegen aligned with your work. Sorry it's taken me so long

@aaliddell aaliddell merged commit 6ed0781 into rules-proto-grpc:master Sep 8, 2023
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation lang-cpp C++ rules specific lang-rust Rust rules specific
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants