Allow union types to have additional fields, support required fields.

[jakemac53]

• Union types can now have additional fields beyond just type and value. These can be passed to each specialized constructor along with the value.
• Adds support for required fields in schemas, which essentially just makes the parameters for these fields in the constructors be required. These are not intended to be widely used, but there are certain things which are truly required.
• Uses these features to add a required id field to request types, and requestId field to response types.
• These will be used to support multiple ongoing requests and remove hacks with global variables for cfe/analyzer internals in the future.
• Also dropped the special _dartWrapperToJson function - afaik this was never necessary (but please let me know if that is wrong).

If you would like, I could separate out the support here from adding the actual required fields.

Risks

This makes requests non-hermetic, because of the id field. This is always only in the outer request type though. We can still do caching based on the inner request type, which I think should work out fine.

Note that having the IDs on the outer types means they are not generally visible to the user APIs which is good.

[davidmorgan]

Nice, thanks, I was thinking we'd need something along these lines :)

A couple of high level thoughts for later, not blocking for this PR:

1. A slightly different way to extend the union types would be to switch from union types to union fields, so a normal object type can have zero, one or more oneOf properties inside it, each of which gets its own type property next to it, generated enum, and pile of is/as methods. I can't think of any obvious reason why we want this, but it's worth bearing in mind in case it looks neater at some point :)

2. All the repetition in the schema is getting annoying, I feel like we are getting gradually closer to wanting to generate the schema too. For example we could flip the generator around, describe the schema in Dart code instead of in JSON and write out both Dart data types and schema. What do you think? If we do that we should also do the TODO about being able to generate and test with a strict schema, at the same time.

[jakemac53]

2. All the repetition in the schema is getting annoying, I feel like we are getting gradually closer to wanting to generate the schema too. For example we could flip the generator around, describe the schema in Dart code instead of in JSON and write out both Dart data types and schema. What do you think? If we do that we should also do the TODO about being able to generate and test with a strict schema, at the same time.

I am fine with the schema for now. It would be nice to be able to write in a bit more of an OOP style with interfaces and such, to remove duplication, but I think it might be over complicating things. The current approach is alright.

It is also hard because extension types can't implement interfaces (other than the representation type), so I am not sure how we would describe the data types and also extend their implementations with all the Dart boilerplate.

[davidmorgan]

2. All the repetition in the schema is getting annoying, I feel like we are getting gradually closer to wanting to generate the schema too. For example we could flip the generator around, describe the schema in Dart code instead of in JSON and write out both Dart data types and schema. What do you think? If we do that we should also do the TODO about being able to generate and test with a strict schema, at the same time.

I am fine with the schema for now. It would be nice to be able to write in a bit more of an OOP style with interfaces and such, to remove duplication, but I think it might be over complicating things. The current approach is alright.

It is also hard because extension types can't implement interfaces (other than the representation type), so I am not sure how we would describe the data types and also extend their implementations with all the Dart boilerplate.

Not sure if I was clear enough to actually convey what I meant :) by "describe the schema in Dart code" I didn't mean real Dart code, which would definitely complicate things, I meant some DSL that we invent to be essentially the JSON schema minus the repetition.

Anyway, no rush, if we need it it will be obvious :) the other thing we may end up doing is adding validation, so you can't write something close to a valid union type but not actually valid, but that seems like more work and possibly less good.

Thanks.

[jakemac53]

Not sure if I was clear enough to actually convey what I meant :) by "describe the schema in Dart code" I didn't mean real Dart code, which would definitely complicate things, I meant some DSL that we invent to be essentially the JSON schema minus the repetition.

Ah ok, we could do that too, but I think it sounds like a lot of work for not a lot of benefit. Any time we want to support something new (for instance, required fields like in this PR), we have to update our special DSL parser etc where I mostly got that for free here.

[davidmorgan]

By "some DSL" I meant the super low effort type of DSL where you just instantiate your data description :) so basically remove the JSON schema parsing and instantiate the data directly.

  generateDartAndSchema([
    JsonClass('Foo', {'foo': JsonType.int, 'bar': JsonType.string}),
    JsonClass('FooBar', {'foo': JsonType.string, JsonType.ref('Foo')}),
    JsonUnion('Baz', [JsonType.string, JsonType.ref('Foo')]),
  ])

Agreed we probably don't want to invent a new type of thing to parse.

Anyway, not urgent.