feat: Use package.links trick for Protobuf compilation

[slowli]

What ❔

Propagates Protobuf information via env variables by relying on package.links.

Why ❔

This simplifies Protobuf configuration and eliminate misconfiguration risks.

[pompon0]

if unused, please remove

[slowli]

Having dependency refs allows ensuring via type system that these dependencies are added in the global pool. Would it be better to force dependencies to get initialized via Lazy::force(), or in some other way?

[pompon0]

Type system doesn't ensure any relation between the dependencies and the newly constructed descriptor, so this is as good as Lazy::force (or just dereferencing the lazy descriptors).

[pompon0]

Would it be hard to lift this constraint?

[slowli]

Theoretically, it can be lifted by changing the exported variable name (manifest rn) depending on the compilation target; but I didn't come up with a good way to ID compilation targets and passing these IDs to dependent crates. IMO, restricting a crate to at most one public target makes sense – it makes sure that all public generated code is in a single place. Do you have use cases for multiple public targets in mind?

[pompon0]

For example roles crate could have a separate protobuf targets for validator and node roles. I don't see any large benefits though, so it can wait.

[slowli]

I should've said from the start: I'm not entirely sure that this approach is better than the one implemented currently.

• (+) It removes requirement to have Protobuf dependencies as build-time deps; this should improve compilation time.
• (+) It also simplifies global descriptor pool management and marginally simplifies Protobuf compilation config.
• (+)The error messages if package.links is forgotten or set to a wrong value should be helpful.
• (-) OTOH, it requires that Protobuf dependencies are direct dependencies, and that they are not renamed. The latter requirement could be lifted with some effort; the former one is fundamental because of how package.links works.

Global descriptor pool management could theoretically be simplified even further by making Descriptors private. It would work by listing all deps (including transitive ones) when a descriptor is created; this is essentially the paths part to Manifest, which we build anyway (and which is topologically sorted by design). A couple of things that bother me about this approach:

• Will the compiler / linker properly deduplicate include_bytes! of the same file?
• Ideally, we'd want to not parse the same descriptor bytes into a descriptor multiple times, which would require keeping track of the loaded descriptors.

The current approach sidesteps both of these issues, and ensures topological sorting of descriptors via Rust type system, which IMO is quite cool.