[ffi] Implement API for interfacing with Calyx modules from Rust

[ethanuppal]

Splits up #2181, making the following changes:

• Adds the tools/calyx-ffi-macro crate for the #[calyx_ffi(...)] procedural macro.
• Adds the tools/calyx-ffi crate, which includes tests that both serve to check the tools/calyx-ffi-macro crate (because the tests compile) and the wrapper introduced in tools/calyx-ffi (because the tests pass), under the assumption that cider2 is fully correct.
• Updates interp/src/lib.rs with a trivial change (that @EclecticGriffin has approved).

[ekiwi]

Thanks for splitting out a smaller chunk! However, I would like you to make an even smaller PR by leaving out the whole calyx_ffi_test machinery. Instead, please just make a PR for everything that is necessary to execute a Calyx design from Rust, afaik, mostly just the code behind the calyx_ffi macro.

I also added some detailed comments.

[ethanuppal]

I don't understand what you mean by the calyx_ffi_test machinery. I can definitely split this into smaller PRs, but the only reason there are two separate crates here is because the Rust requires procedural macros to go into separate crates. The code in both crates are tightly intertwined, so I don' know if it makes sense to split it up.

[ekiwi]

I don't understand what you mean by the calyx_ffi_test machinery.

What I mean is that the calyx_ffi macro does not depend on the calyx_ffi_tests macro, and it can be used in isolation. Is that correct? If so, I would like to review a PR that only adds the calyx_ffi macro and does not include the calyx_ffi_tests macro and related code.

[ethanuppal]

Oh, I see. Got it.

[ethanuppal]

The test are dependent on this macro though, so this would mean we merge it in with no tests? Or would we add back the tests with the calyx_ffi_test macro promptly?

[ekiwi]

The test are dependent on this macro though, so this would mean we merge it in with no tests? Or would we add back the tests with the calyx_ffi_test macro promptly?

Can you write a couple of "bare metal" tests that do not use the calyx_ffi_test macro?

[ethanuppal]

I think it should actually be fine, we can just merge the tests back in with the next PR.

[ekiwi]

Thanks for splitting up the PR. The current size is much more manageable.

I added some small comments, but my big question is the following:

Why does the backend have to be specified as part of the calyx_ffi macro? If you look at Serde, you apply their macro to your struct without specifying the backend that you are going to use. The #[derive(Serialize)] just implements the correct trait such that you can then serialize the struct with all the available Serde backends.

Could we do a similar thing here? I.e. could we have something like:

#[calyx_ffi(
    src = "tests/adder.futil",
    comp = "main",
)]
struct Adder;

let adder_sim = Adder::start_sim(CiderSimulator);

CiderSimulator would receive the name of the calyx file and component, generate a generic simulator which would then be saved inside of the Adder struct. Adder could even be generic on the type of backend used so that you can avoid any overhead from dynamic dispatch. I.e., the macro generated code would have something like:

struct Adder<B: SimBackend> {
  // ....
  sim: B, // could also be Option<B> if you want to allow instantiating `Adder` without a backend
}

trait SimBackendFactory { // this is implemented by CiderBackend
  type Output;
  fn create(path, component_name, ...) -> Output;
}

impl <B: SimBackend> Adder {
  fn start_sim<F: SimBackendFactory<Output = B>>(f: &F) -> Self {
    let sim = f.create(..., ...);
    Self { sim }
  }
}

[ethanuppal]

@ekiwi The reason is that I need to know the port field names. I need to be able to hook up port field names with the backends. I agree that there must be a nicer way to do it, but I can't think of one. Also, I have a CalyxFFI object from which you create these from so you don't reload the same calyx file multiple times.

[ethanuppal]

Are we good to merge?