re-use definitions from host functions in fp_import

[joepio]

In the examples dir, we have the my_complex_imported_function defined two times: in the original bindings.rs and in the host's own logic, somewhere (in this case spec/mod.rs). The danger of having two definitions in source code, is that both can get out of sync.

I'd like to re-use the hosts's function definition in fp_import, in some way, or maybe have some type-check system that prevents those from getting out of sync. I was thinking of using a Trait for this (define a Host or something), but I'm not sure if that's the way to go.

Or maybe I'm missing something obvious. Looking forward to your thoughts!

EDIT: wait, maybe I can simply define my host functions inside fp_import... Is that the obvious way to go I'm missing?

EDIT 2: No, I can't export function from within fp_import to use by the host app.

[arendjr]

I like this request! I don’t think you need to worry too much about them getting out-of-sync (it shouldn’t compile otherwise), but I agree it could be convenient if one of the implementations could simultaneously act as protocol definition.

For our own frontend we use a lot of Rust core logic in a component we call FiberKit. We use fp-bindgen there too, even though there is only one “plugin” and one runtime (the TypeScript). So there we kinda have the inverse problem: It would be nice if the implementation of the plugin could act as definition of the API. It’s not really a hindrance in our day-to-day work to have the protocol separate, but it does complicate the setup somewhat.

However, while in both cases it would be nice to have, I’m not entirely sure how to implement this. Regardless of whether it’s the implementation of the runtime or the plugin, they link against their respective generated bindings. But if they also need to act as source of truth from which to generate the bindings, you run into a chicken-egg problem: How do you generate the bindings if the code from which they are generated requires the bindings themselves in order to compile?

[joepio]

How do you generate the bindings if the code from which they are generated requires the bindings themselves in order to compile?

Maybe I'm misunderstanding, but is it possible to read the function definitions from the spec/mod.rs file (i.e. the place where the host's functions are defined), and use these? They do not depend on the generated bindings, right? I'm not familiar with programming macros, but if they have access to external functions, this may work, right?

A second solution may be to let host define a Trait which implements a bunch of functions, and re-use these.

[arendjr]

Well, technically it’s possible, but it’s either very complex to do so or it comes with some very unintuitive caveats. The problem is that we use Rust macros to extract the protocol information, and for the macros to work the code in which the macros reside also needs to compile. But Rust source files are not compiled individually, they’re part of a crate that either compiles as a whole or doesn’t. The crate is what provides a source file with context about module paths and dependencies that are available.

So, in order to extract a protocol definition using macros from the spec/mod.rs file, the entire crate in which the spec module resides must compile.

Right now, I see two ways around this:

1. Don’t use macros inside the file, but parse the file through some external mechanism (could be a macro from another file). This does add significant complexity however, because when interpreting the information from that file, we can no longer rely on the module resolution from the Rust compiler, because we our own code is no longer running within the context of the same module. It might be possible to get to work, but I’m afraid it would add significant complexity.
2. The other workaround I see is to create a symlink to spec/mod.rs and evaluate it from another crate that does not depend on the generated bindings. This might be possible, but it would be very unintuitive to whoever’s implementing that file, because they need to consider that any module paths they use need to be valid from the perspective of both crates. Even just documenting such gotchas gives me the shivers :)