Skip to content

Emit json.Marshaller/Unmarshaller implementations using jsonpb #926

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
mitchellh opened this issue Aug 11, 2019 · 2 comments
Closed

Emit json.Marshaller/Unmarshaller implementations using jsonpb #926

mitchellh opened this issue Aug 11, 2019 · 2 comments

Comments

@mitchellh
Copy link

mitchellh commented Aug 11, 2019

Is your feature request related to a problem? Please describe.

We are using protobufs heavily internally for systems that require JSON-compatible wire formats. We typically embed proto.Message implementations within standard Go structs which are then JSON-encoded using encoding/json. This doesn't work with most protobuf messages for the same reasons jsonpb was created in the first place.

The json-encoding is dictated by a 3rd party system we don't control. It provides hooks for a custom encoder/decoder but we'd like to avoid changing something that core to the system if possible.

Describe the solution you'd like
What we'd like is to have protoc-gen-go emit json.Marshaler/Unmarshaler implementations that use jsonpb so that if you use encoding/json it automatically emits the correct format.

This can potentially be implemented as a built-in plugin so it is optional with no backwards compatibility issues. Then all we have to do is plugins=jsonpb or something. That would be great too.

This is particularly important given the problem statement above where we're embedding concrete proto.Message types within other Go structs and we'd like to encode them all as JSON.

Describe alternatives you've considered
A few things:

  • We originally switched to []byte in the Go structs and did the jsonpb-roundtrip ourself. This results in significant boilerplate and encouraged us to look at other solutions.
  • We then planned to have two formats: either pure Go (encoding/json) or pure proto.Message (struct defined in proto) then using a type switch to encode/decode using only encoding/json or jsonpb. This was in progress until the next...
  • We've written our own protoc- binary that emits the json.Marshaler/Unmarshaler implementation for every message to use jsonpb.

I'd love to implement a protoc-gen-go plugin but that requires it to be compiled in as far as I can tell and we'd like to not maintain a fork!

Additional context
jsonpb.Marshaler exposes a number of configurables. Right now we're just using the defaults, but we plan on exposing proto annotations to control these by package, file, message, etc. This would probably be important for this too.

@dsnet
Copy link
Member

dsnet commented Aug 12, 2019

Duplicate of #256 for protoc-gen-go to provide this feature as an option.

This can potentially be implemented as a built-in plugin so it is optional with no backwards compatibility issues. Then all we have to do is plugins=jsonpb or something.

The idea of a separate generate to add these methods is interesting.

The concept of a protoc-gen-go "plugin" is going away in v2 as it is not maintainable since there lacks any specified agreement between the main generator and the plugin on how to cooperate and not step on each others toes. In v2, we will instead be exposing a compiler/protogen, which makes it easy to write plugins to protoc itself.

As an experiment, I quickly wrote a protoc plugin that generates the {Marshal,Unmarshal}JSON methods: https://golang.org/cl/189918

jsonpb.Marshaler exposes a number of configurables. Right now we're just using the defaults, but we plan on exposing proto annotations to control these by package, file, message, etc. This would probably be important for this too.

If we generate the MarshalJSON and UnmarshalJSON methods. How would that work with custom options? Those methods have no ability to recursively plumb custom options downwards.

@dsnet dsnet closed this as completed Aug 12, 2019
@mitchellh
Copy link
Author

Sorry I searched and still didn't find the duplicate! I'll comment there.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants