Round-trippable cty type serializations

[apparentlymart]

The initial focus of this package was on marshaling and unmarshaling the subset of type/value concepts that cty and YAML have in common, accepting that type information can therefore be lost on any round-trip.

It may also be desirable to support an opt-in (explicitly via ConverterConfig) mode where the Marshal function generates cty-specific YAML tags and the Unmarshal and ImpliedType functions honor them to permit exact round-tripping with intact type information.

%TAG !cty! (some URL prefix TBD)
---
foo: !cty!object
  bar: !cty!string "Hello World"
  baz: !cty!null?of=list(string) ~
bar: !cty!set
  - !cty!string "a"
  - !cty!string "b"
  - !cty!string "c"

The Unmarshal and ImpliedType methods should both ignore these custom tags by default, unless explicitly opted in via ConverterConfig, so that existing applications using this library as a generic YAML decoder are not surprised by suddenly seeing types outside of the cty subset they are expecting.

Perhaps during marshaling we'd omit the tags that add no additional information compared to standard YAML, such as omitting !cty!string on strings and just emitting quoted and literal strings always, and omitting !cty!object and !cty!tuple on mappings and sequences because those are the default interpretations of those constructs anyway. The decoder ought to still recognize those explicit tags though, for completeness and consistency.

%TAG !cty! (some URL prefix TBD)
---
# Variant with explicit tags only where they add new information
foo:
  bar: "Hello World"
  baz: !cty!null?of=list(string) ~
bar: !cty!set
  - "a"
  - "b"
  - "c"

---

A further extension of this cty-annotated YAML serialization would be to support unknown values, in a similar way to how we represented null in the above:

%TAG !cty! (some URL prefix TBD)
---
foo: !cty!unknown?of=string ~

If we support this, it should be via a separate opt-in that is in addition to the one for supporting the other tags, so applications can control whether they will accept unknown values. This is consistent with cty's promise that unknown values are always opt-in and so an application can avoid having to deal with them if it never introduces them. (cty msgpack is already inconsistent with that principle, which is unfortunate, but in that case its primary reason to exist over JSON is to serialize unknowns, so perhaps reasonable.)

If the decoder encounters an unknown value when not opted in, that would be an Unmarshal or ImpliedType error. If the encoder encounters an unknown value when not opted in, then just like today it would refuse to encode it with an error.

Note that an explicit serialization of the dynamic pseudo-type as a tag is not necessary because during encoding its precise type would be stored via the type-oriented tags, and can thus be recovered on decoding. However, we would need to be able to talk about null and unknown values of the dynamic pseudo-type, which would require it to be supported in the of= argument on the null and unknown tags:

• !cty!null?of=any
• !cty!unknown?of=list(any)

There can never be a known, non-null value of the dynamic pseudo-type (that's why it's a "pseudo-type") so it does not need its own specific type tag.

---

A serialization with these extra tags is likely not consumable my most standard YAML parsers, so it's not clear yet whether such a thing is even useful (the JSON and msgpack serializations already provide round-trippability, and YAML syntax is primarily for human-to-machine interactions rather than machine-to-machine anyway) so the above may never be implemented, but I wanted to capture the idea so that there'd be somewhere to discuss any use-cases that arise for it.

Since this would add quite a bit of complexity to the implementation, I think the bar to be met here would be a use-case that cannot already be met with JSON or msgpack, for some good reason other than personal taste for YAML syntax over JSON syntax.