ApiDOM converter

[char0n]

OpenAPI Changelog

I would recommend Mike Ralphson’s article describing OpenAPI 3.1.0 changes comparing to OpenAPI 3.0.x, which I found to be best source of truth related to changes between the relevant versions. It’s even better than official OpenAPI 3.1.0 release log. Another source of possibly valuable info: https://github.com/apiture/openapi-down-convert. The spotlight article also contains wrong information about JSON Schema version used in OpenAPI 3.1.0. The version used in OpenAPI 3.1.0 is actually JSON Schema Draft 2020-12 and not 2019-09.

Target versions

We're targeting OpenAPI 3.0.3 (latest released version of OpenAPI 3.0 release branch). OpenAPI 3.0.4 is currently in making.

Doing conversion on resolved definition

To clarify terms a bit for future reference: “Resolved” wording is used in original Swagger JavaScript tooling to refer to dereferencing. Dereferencing is part of the nomenclature used by JSON Schema and ApiDOM and refers to the mechanism of reference removal. In JSON Schema and ApiDOM context “resolve” refers more to how URIs are resolved and retrieved.

For converted to work the dereferenced OpenAPI 3.1.0 would have to eliminate possible cycles so that the dereferenced OpenAPI 3.1.0 can be easily traversed and converted to OpenAPI 3.0.3 and serialized in YAML 1.2 or JSON. Since JSON Schema 2019-09 the reference removal is no longer safe or possible. Which means ApiDOM dereferencing mechanism is just an approximation of the ideal state.

Instead of dereferencing, the more suitable mechanism for creating a compound document would be bundling. ApiDOM has a bundling framework implementation. OpenAPI 3.1.0 bundling strategy have to be implemented and plugged into the bundling framework. Bundling is codified by its own spec for JSON Schema 2020-12 while there is no such spec for OpenAPI and we would go with an opinionated processing. We could base that part on existing bundling done by Java projects. The work on bundling and the converter logic could proceed in parallel. Fixtures used for converter implementation would assume bundled versions of OpenAPI 3.1 definitions.

Deficiencies in current ApiDOM OpenAPI 3.1.0 bundling implementation

1. missing support for jsonSchemaDialect

OpenAPI.jsonSchemaDialect is supported but its support is currently limited to not being defined at all or to be defined with “https://spec.openapis.org/oas/3.1/dialect/base”. This effectively means that we treat every JSON Schema Object as if it would be JSON Schema Draft 2020-12 with OpenAPI 3.1.0 specific dialect.

2. missing support for “Differing and Default Dialects”

What “Differing and Default Dialects” means is directly described in JSON Schema: A Media Type for Describing JSON Documents . In human language - this means that we basically ignore the $schema keyword and assume every JSON Schema Object is of “https://spec.openapis.org/oas/3.1/dialect/base” dialect.

3. missing support for $dynamicRef and $dynamicAnchor

These JSON Schema 2020-12 keywords are not supported at all. So if anybody references a JSON Schema with it, we’re just ignoring it.

4. missing support for relative JSON Pointers

ApiDOM now supports parsing, compiling and evaluating Relative JSON Pointers that are part of JSON Schema 2020-12. Unfortunately this support has not be added to our dereferencing mechanism yet.

5. error tolerance

By default all parsing (which is done during bundling) including parsing of remote references are done using default ApiDOM parser adapters that use tree-sitter. This tree-sitter is a parser generator which is very tolerant to various error in JSON or YAML 1.2 formats and will parse even incomplete and invalid documents.

Converter logic implementation options

1. ApiDOM structure is traversed with ApiDOM traverse functions and its elements updated inline accordingly when 3.1 constructs need to be transformed into 3.0. This allows to make use of semantic provided by ApiDOM + JSON/YAML serialization also provided by ApiDOM

2. ApiDOM structure is traversed with ApiDOM traverse functions but output if freely built by the converter logic e.g. directly in JSON. This probably makes more sense when the outcome is a structure completely different from the original one, so not in this case.

3. Converter logic is given as input a JSON object (obtained from ApiDOM) to be traversed with whatever logic. This means that ApiDOM is not part of the game after bundling. It's an option but we are not taking advantage of ApiDOM semantic and other capabilities.

Important

It is recommended to go with option 1.)

---

Architectural design

The overall package architecture SHOULD have the same interface as the apidom-reference package. First reason is that converter will use OpenAPI 3.1.0 bundler strategy from the apidom-reference and associated bundling code infra. Second reason is API consistency.

First iteration of the converter will use saturated (default) apidom-reference exports. This will gets optimized later - only needed components from apidom-reference will be used and empty apidom-reference export MUST be used.

convert and convertApiDOM functions will be exported from the apidom-converter package. Both functions will return ApiDOM, specifically ParseResultElement.

ParseResultElement.prototype.api or ParseResultElement.prototype.result will return the converted API definition. ParseResultElement.prototype.annotations will return all collected annotations
ParseResultElement.prototype.warnings will return all collected annotations with warning meta class.
ParseResultElement.prototype.errors will return all collected annotations with error meta class.

Under the hood these functions will use strategy pattern based on standardized interface of ConverterStrategy. This will allow to create custom strategies that may or may not even use ApiDOM. But our primary goal is to use the ApiDOM and use it's capacity for the conversion.

I've implemented initial OpenAPI31ToOpenAPI30ConvertStrategy, which does the actual conversion from OpenAPI 3.1.0 to OpenAPI 3.0.3. The conversion has two steps:

1. First the individual refractor plugins are dispatched to provide specific conversion from OpenAPI 3.1.0 to OpenAPI 3.0.3
2. Next the ApiDOM resulted from specific conversion is run via OpenAPI 3.0.3 refractor to convert semantically all elements in ApiDOM to OpenAPI 3.0.3

Refractor plugins can either perform mutable or immutable transformation. I would recommend to go for immutable. If it is decided to perform mutable transformations, the bundled ApiDOM needs to be deep cloned before it can be manipulated.

Annotations are created in-place, meaning they need to be created immediately, when we do any ApiDOM manipulation, like removing or adding new elements. Annotations SHOULD be assigned source maps of original element whenever possible.

webhooks refractor plugin demostrates how to use annotations.

Resources

• unordered list of refs + concepts here
• https://github.com/apiture/openapi-down-convert