Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support maps for definitions #219

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

sasa1977
Copy link
Contributor

This is the second optimization mentioned in #217. As you proposed there, the new code still works with the current format (list). Note that I was deliberately sloppy with the typespec for the new format, because it would require a lot of duplication with the existing def.

Personally, I think that going forward the new format should be preferred. In this case, we could provide a detailed spec for the map, while using the relaxed list() spec for the "legacy" format.

Of course, we could always have two detailed specs, but maintaining this is going to be more difficult and error prone.

I don't have a strong opinion though, so let me know what you prefer.

sasa1977 added 2 commits July 11, 2022 10:39
Passing indexed definitions improves the performance of the encoder.
Copy link
Owner

@tomas-abrahamsson tomas-abrahamsson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks fine, but I have a few concerns:

  • Could you add a function to convert list-defs to a map, since it is maybe not always obvious: the .proto, may result in 3-element tuples for options in some cases, if I remember right, but maps:to_list() would choke on that. How did you do this? Filter for 2-tuples and then maps:to_list? Maybe such a function should go either into gpb_defs.erl or maybe into gpb.erl for now, it don't know what's best. Maybe gpb.erl for now, as the code generator doesn't need maps that much.
  • Does the merge_msgs/3 handle maps? During decoding, if the bytes on the wire contains a field twice, the field is to be merged as mandated by the protobuf. For scalars. merging means overwriting, for repeated it means appending, but if the field is a sub-message, merging means recursively merging the fields of the sub-messages. Maybe this is handled already? I haven't digged deep enough yet, but wanted to make you aware of it.
  • It would be good with some unit test with a map. Even if the code is correct now, unit tests also makes sure the functionality doesn't get accidentally removed in the future for example in a refactoring. Beware of the somewhat unorthodox structure of the unit tests though: your tests would be in gpb_tests.erl, but gpb_compile_tests.erl actually -include()s the gpb_tests.erl. Yes, the .erl file. The background for this is that most tests for the data-driven encoder/decoder should work also with the generated encoder/decoder. The top and bottom of the gpb_tests.erl contains code that is not shared with gpb_compile_tests.erl. That would be a good place to put your tests.

@@ -46,7 +46,7 @@

-define(is_non_empty_string(Str), (is_list(Str) andalso is_integer(hd(Str)))).

-type defs() :: [def()].
-type defs() :: [def()] | map().

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm ok with | map() being a bit unspecified for the time being, but I would like the | map() to be added to the relevant function in gpb.erl instead of here, since not every function that accept defs() can handle a map().

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants