Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better proc-macro for Step enums #771

Merged
merged 9 commits into from
Sep 19, 2023
Merged

Better proc-macro for Step enums #771

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
25be6e4
Better step macro
taikiy Jul 21, 2023
c66a842
use enum's ident as error span
taikiy Aug 10, 2023
74bf4df
Remove unnecessary derives from #step expansion
taikiy Aug 11, 2023
2c92ff8
Merge remote-tracking branch 'upstream/main' into new_gate_macro
taikiy Aug 11, 2023
92ff7c0
update comments and a minor cosmetic change
taikiy Aug 11, 2023
54db559
Merge branch 'main' into new_gate_macro
taikiy Aug 15, 2023
df0a50f
Merge branch 'main' into new_gate_macro
taikiy Sep 14, 2023
eab864a
remove osbolete attribute
taikiy Sep 14, 2023
dbbbcde
fix clippy format error
taikiy Sep 14, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 1 addition & 4 deletions Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ step-trace = ["descriptive-gate"]
# The following two features are mutually exclusive. Descriptive should be enabled by deafult as the vast majority
# of unit tests use it. Compact uses memory-efficient gates and is suitable for production.
descriptive-gate = []
compact-gate = []
compact-gate = ["ipa-macros/compact-gate"]

# Standalone aggregation protocol. We use IPA infra for communication
# but it has nothing to do with IPA.
Expand Down Expand Up @@ -82,14 +82,11 @@ serde = { version = "1.0", optional = true, features = ["derive"] }
serde_json = { version = "1.0", optional = true }
sha2 = "0.10.6"
shuttle-crate = { package = "shuttle", version = "0.6.1", optional = true }
strum = {version = "0.25", features = ["derive"] }
thiserror = "1.0"
time = { version = "0.3", optional = true }
tinyvec = "1.6"
tokio = { version = "1.28", features = ["rt", "rt-multi-thread", "macros"] }
tokio-rustls = { version = "0.24.0", optional = true }
tokio-stream = "0.1.14"
tokio-util = "0.7.8"
toml = { version = "0.7", optional = true }
tower = { version = "0.4.13", optional = true }
tower-http = { version = "0.4.0", optional = true, features = ["trace"] }
Expand Down
124 changes: 0 additions & 124 deletions ipa-macros/Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

16 changes: 4 additions & 12 deletions ipa-macros/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,12 @@ version = "0.1.0"
rust-version = "1.64.0"
edition = "2021"

[lib]
proc-macro = true

[features]
trybuild = []
compact-gate = []

[[test]]
name = "tests"
path = "tests/mod.rs"
required-features = ["trybuild"]
[lib]
proc-macro = true

[dependencies]
syn = { version = "2.0.15", features = ["full"] }
syn = { version = "2.0.15", features = ["full", "extra-traits"] }
quote = "1.0.27"

[dev-dependencies]
trybuild = { version = "1.0.80", features = ["diff"] }
84 changes: 5 additions & 79 deletions ipa-macros/src/derive_gate/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,62 +2,11 @@ use proc_macro::TokenStream;
use quote::quote;
use syn::{parse_macro_input, DeriveInput};

use crate::parser::{group_by_modules, ipa_state_transition_map, module_string_to_ast};
use crate::parser::{group_by_modules, ipa_state_transition_map};

// Procedural macro to derive the Step and StepNarrow traits and generate a memory-efficient gate.
//
// The goal is to generate a state transition graph and the corresponding `StepNarrow` implementations
// for the IPA protocol. This macro assumes that a complete IPA steps file exists in the repo at the
// location specified as `STEPS_FILE`. The steps file can be generated by running `collect_steps.py`.
//
// The steps file contains a list of narrowed steps, where each line represents a hierarchy of narrowed
// steps delimited by "/". For example, the following lines represent a hierarchy of narrowed steps:
//
// RootStep => 0
// RootStep/StepA::A1 => 1
// RootStep/StepA::A1/StepB::B1 => 2
// RootStep/StepA::A1/StepB::B2 => 3
// RootStep/StepC::C1 => 4
// RootStep/StepC::C1/StepD::D1 => 5
// RootStep/StepC::C1/StepD::D1/StepA::A2 => 6
// RootStep/StepC::C2 => 7
//
// From these lines, we want to generate StepNarrow implementations for each step.
//
// impl StepNarrow<StepA> for Compact {
// fn narrow(&self, step: &StepA) -> Self {
// Self(match (self.0, step.as_ref()) {
// (0, "A1") => 1,
// (5, "A2") => 6,
// _ => panic!("invalid state transition"),
// })
// }
// }
// impl StepNarrow<StepB> for Compact {
// fn narrow(&self, step: &StepB) -> Self {
// Self(match (self.0, step.as_ref()) {
// (1, "B1") => 2,
// (1, "B2") => 3,
// _ => panic!("invalid state transition"),
// })
// }
// }
// ...
//
//
// Currently, this derive notation assumes it annotates the `Compact` struct defined in
// `src/protocol/step/compact.rs`. The `Compact` struct is a wrapper around a `u16` value that
// represents the current state of the IPA protocol.
//
// In the future, we might change the macro to annotate each step in the IPA protocol. The macro
// will then generate both `Descriptive` and `Compact` implementations for the step. However, that
// kind of derive macro requires more annotations such as the fully qualified module path of the
// step. This is because there are many locally-defined `Step` enums in IPA, and we need to
// disambiguate them. However, proc macro doesn't receive the fully qualified module path of the
// annotated struct.
/// Generate a state transition graph and the corresponding `AsRef<str>`
/// and `deserialize()` implementations for `Compact` gate.

/// Generate a state transition graph and the corresponding `StepNarrow` implementations for the
/// IPA protocol.
pub fn expand(item: TokenStream) -> TokenStream {
// `item` is the `struct Compact(u16)` in AST
let ast = parse_macro_input!(item as DeriveInput);
Expand All @@ -67,39 +16,16 @@ pub fn expand(item: TokenStream) -> TokenStream {
_ => panic!("derive Gate expects a struct"),
}

// we omit the fully qualified module path here because we want to be able to test the macro
// using our own implementations of `Step` and `StepNarrow`.
let mut expanded = quote!(
impl Step for #gate {}
impl crate::protocol::step::Step for #gate {}
);

let steps = ipa_state_transition_map();
let grouped_steps = group_by_modules(&steps);
let mut reverse_map = Vec::new();
let mut deserialize_map = Vec::new();

for (module, steps) in grouped_steps {
// generate the `StepNarrow` implementation for each module
let module = module_string_to_ast(&module);
let states = steps.iter().map(|s| {
let new_state = &s.name;
let new_state_id = s.id;
let previous_state_id = s.get_parent().unwrap().id;
quote!(
(#previous_state_id, #new_state) => #new_state_id,
)
});
expanded.extend(quote!(
impl StepNarrow<#module> for #gate {
fn narrow(&self, step: &#module) -> Self {
Self(match (self.0, step.as_ref()) {
#(#states)*
_ => static_state_map(self.0, step.as_ref()),
})
}
}
));

for (_, steps) in grouped_steps {
// generate the reverse map for `impl AsRef<str> for Compact`
// this is used to convert a state ID to a string representation of the state.
reverse_map.extend(steps.iter().map(|s| {
Expand Down
Loading