Request for Feedback: OCI image-spec 1.1.0

[sudo-bmitch]

OCI is approaching a 1.1.0 release on the image-spec and distribution-spec, so we are reaching out to implementations to see if you have any concerns implementing support for those changes that we should address before a GA release.

A full diff of the changes can be viewed at:

• image-spec: opencontainers/image-spec@v1.0.2...opencontainers:image-spec:main
• distribution-spec: opencontainers/distribution-spec@v1.0.1...main

I'm happy to take feedback here, you can raise an issue in OCI, or respond in our tracking issue at opencontainers/image-spec#1093.

[mtrmac]

@sudo-bmitch Looking at the distribution spec so far, I have filed opencontainers/distribution-spec#454 . I don’t think that’s blocking the spec release at all.

---

Note to self - distribution spec changes:

• A new possibility to mount a blob without specifying the source repo. “MAY” be supported; failure still initiates an upload. Probably not worthwhile right now, worth at least a comment in code. Possibly relevant for docker, BlobInfoCache: try to reuse blobs from set of all known compressed blobs when pushing across registries #1645 .
• New referrers list endpoint: https://github.com/opencontainers/distribution-spec/blob/6bc87156eacf3b73362db343eb6b63d7abeedf7e/spec.md#listing-referrers (writes are implied by manifest uploads). Allows filtering by a single MIME type (at least Support OCI Image Manifests with artifactType and subject properties distribution/distribution#3834 suggests only one MIME type).
• Compatibility: emulating subject support via a tag convention.
  • Required (MUST) to emulate when uploading a manifest
  • Required (MUST) to emulate when querying for referrers
  • Recommended (SHOULD) to emulate support when deleting a manifest
• “If a client receives Warning response headers, it SHOULD report the warnings to the user in an unobtrusive way.
  Clients SHOULD deduplicate warnings from multiple associated responses.”

Other spec points of note:

• (Recorded elsewhere) Deleting a tag, not a manifest by digest, is possible
• “A 4XX response code from the registry MAY return a body in any format” (contra the traditional docker/distribution implementation), but if it is JSON, there is a mandatory format.

[mtrmac]

Note to self — image spec changes:

• New os.* and variant, ArgsEscaped fields in image config — probably mostly irrelevant, but see e.g. checkImageDestinationForCurrentRuntime. OTOH Podman might need to update how it creates runtime-spec data, per https://github.com/opencontainers/image-spec/blob/main/conversion.md : runtime-spec conversion does not follow current OCI spec podman#19459 .
• data property in descriptors — probably consume it, and possibly produce it (with what heuristics?)
• artifactType property in descriptors — seems irrelevant right now
• subject field on both manifests and image indices.
• artifactType in manifests; “ the config.mediaType is set to the empty value, the artifactType MUST be defined.”
• “Any extra fields in the Image JSON struct are considered implementation specific and MUST NOT generate an error by any implementations which are unable to interpret them.” - probably worth adding a test. (OTOH they are also ignored on format conversions)
• “Implementations processing content SHOULD NOT generate an error if they encounter an unknown property in a known media type.” [Sure, but to be explicit that data is just lost on edits.] Probably worth adding tests, both for parsing in general, and possibly that no-edit copies don’t lose that data.
• (Entries in index’manifests:) “An encountered mediaType that is unknown to the implementation MUST NOT generate an error.”
• (Manifest itself:) “An encountered mediaType that is unknown MUST NOT generate an error.”
• “Implementations storing or copying image manifests MUST NOT error on encountering a [config MIME type] value that is unknown to the implementation.”, and “MUST NOT attempt to parse the referenced content if this media type is unknown”
• Manifests’ layers “SHOULD have at least one entry”, i.e. we should not break on 0 (but there MUST be a layer on on a non-artifact)
• Manifests’ layers “Implementations storing or copying image manifests MUST NOT error on encountering a mediaType that is unknown to the implementation.”
• application/vnd.oci.empty.v1+json for “unused descriptors” — we probably don’t need to treat that specially.

Other spec points of note:

• “JSON content SHOULD be serialized as canonical JSON”
• specs-go/v1/Image{LayoutFile,LayoutVersion,IndexFile,BlobsDir should be used in the oci* transports.

[mtrmac]

@sudo-bmitch Looking at containers/image, I think the changes are very useful and I have no concerns with the spec.

It will take some time for the “MUST” compatibility mechanisms on manifest upload / deletion to be implemented, though — both to just implement them (they don’t exist right now in c/image), for the implementations to propagate to supported products, and ultimately for users to upgrade their stable systems which can already upload OCI manifests. So I’m not sure that the “MUST” requirements, in practice, provide a guarantee any other part of the ecosystem can rely on, outside of repos with strictly controlled writers (which don’t really EDIT need this world-wide spec guarantee to impose that strict control).

[cgwalters]

I think the idea is it's basically "MUST" if one claims/advertises 1.1 support/compliance.

[sudo-bmitch]

I think the idea is it's basically "MUST" if one claims/advertises 1.1 support/compliance.

That's exactly it. The "MUST" sections turn into a conformance test for registries to advertise support for 1.1 conformance. For clients, it's good to ensure you have support for older and non-conformant registries.

[mtrmac]

#2062 fixes the low-hanging fruit.

More is tracked by RH as https://issues.redhat.com/browse/RUN-1880 .

[davidspek]

distribution/distribution is waiting for the release to be cut before implementing the changes (see distribution/distribution#3834 (comment)).