Support $yaml and $json fields in API Kind definitions

[daftgopher]

Describe Request:

When validating entries with Kind: API, the validator only checks for a string type on the spec.definition field at the moment. The backstage model however has added limited support for object types on the definition field when the keys are one of the following:

• $json
• $yaml
• $text

Examples:

From the Backstage Documentation

apiVersion: backstage.io/v1alpha1
kind: API
metadata:
  name: petstore
  description: The Petstore API
spec:
  type: openapi
  lifecycle: production
  owner: [email protected]
  definition:
    $text: https://petstore.swagger.io/v2/swagger.json

Supporting Documentation Links:

https://backstage.io/docs/features/software-catalog/descriptor-format#substitutions-in-the-descriptor-format

[roadie-bot]

https://app.shortcut.com/larder/story/9176

[daftgopher]

Amending this slightly because I found that $text will actually pass the validator. $yaml and $json still do not seem to be supported.

[kissmikijr]

Hey, it is confusing, however the linked docs generally talks about the addition of these processors to Backstage, and gives an example with an entity Kind: API.
But if you try the catalog does not accept anything else to the definition field only string. You can see the schema that we use to validate here: https://github.com/backstage/backstage/blob/490d663a55ad1d91850979048b0582056e45517b/packages/catalog-model/src/kinds/ApiEntityV1alpha1.ts#L37

In the case of these substitutions the preprocessor would create a json object to the definition, but the definition should be a json string. That's why it fails.

[daftgopher]

Yeah, I have a little more context on this now and I realize that the processor essentially reads and parses the contents of these $-prefixed keys as embedded strings during the catalog ingestion process. However, I've also noticed that $json and $yaml either don't work the way I expect or I'm simply using them incorrectly because I've never gotten them to properly fetch data from a remote source, e.g.

kind: API
...
spec:
  ...
  definition:
    $yaml: https:///www.example.com/path/to/an/api/definition.yaml # Does not inline into a string

This is probably why they're failing to inline correctly and the validator is treating them as objects. I've never actually seen these work when I try examples like the above. Is it correct to assume the validator is running some kind of code to inline $text as a string before the validation check is done? If so, and that code is also inlining $yaml and $json as intended, then this may be ok to close.

[sikemausa]

To piggyback off of this one, running into the same issue, but whilst trying to implement an OpenAPI spec.

spec:
  type: type
  system: system
  lifecycle: lifecycle
  owner: owner
  definition:
    $openapi: ./docs/api/oas.yaml

The above will fail validation with the following error:

Failed to validate api/catalog-info.yaml: TypeError: /spec/definition should be string - type: string

I have verified that this does indeed work within Backstage whilst pointing to an OpenAPI spec