diff --git a/docs/v3/codecs.rst b/docs/v3/codecs.rst index 0bb2536..80a373a 100644 --- a/docs/v3/codecs.rst +++ b/docs/v3/codecs.rst @@ -2,12 +2,15 @@ Codecs ====== -Under construction. +Core codecs +----------- + +The following documents specify the core codecs. Read more about core codecs in the :ref:`core codecs ` section of the Zarr version 3 specification. .. toctree:: :glob: :maxdepth: 1 :titlesonly: - :caption: Contents: + :caption: Core codecs: - codecs/*/* + codecs/core/*/* \ No newline at end of file diff --git a/docs/v3/codecs/blosc/v1.0.rst b/docs/v3/codecs/core/blosc/v1.0.rst similarity index 100% rename from docs/v3/codecs/blosc/v1.0.rst rename to docs/v3/codecs/core/blosc/v1.0.rst diff --git a/docs/v3/codecs/bytes/v1.0.rst b/docs/v3/codecs/core/bytes/v1.0.rst similarity index 100% rename from docs/v3/codecs/bytes/v1.0.rst rename to docs/v3/codecs/core/bytes/v1.0.rst diff --git a/docs/v3/codecs/crc32c/v1.0.rst b/docs/v3/codecs/core/crc32c/v1.0.rst similarity index 100% rename from docs/v3/codecs/crc32c/v1.0.rst rename to docs/v3/codecs/core/crc32c/v1.0.rst diff --git a/docs/v3/codecs/gzip/v1.0.rst b/docs/v3/codecs/core/gzip/v1.0.rst similarity index 100% rename from docs/v3/codecs/gzip/v1.0.rst rename to docs/v3/codecs/core/gzip/v1.0.rst diff --git a/docs/v3/codecs/sharding-indexed/sharding.png b/docs/v3/codecs/core/sharding-indexed/sharding.png similarity index 100% rename from docs/v3/codecs/sharding-indexed/sharding.png rename to docs/v3/codecs/core/sharding-indexed/sharding.png diff --git a/docs/v3/codecs/sharding-indexed/v1.0.rst b/docs/v3/codecs/core/sharding-indexed/v1.0.rst similarity index 100% rename from docs/v3/codecs/sharding-indexed/v1.0.rst rename to docs/v3/codecs/core/sharding-indexed/v1.0.rst diff --git a/docs/v3/codecs/transpose/v1.0.rst b/docs/v3/codecs/core/transpose/v1.0.rst similarity index 100% rename from docs/v3/codecs/transpose/v1.0.rst rename to docs/v3/codecs/core/transpose/v1.0.rst diff --git a/docs/v3/core/v3.0.rst b/docs/v3/core/v3.0.rst index 4f8d256..0ab3cf2 100644 --- a/docs/v3/core/v3.0.rst +++ b/docs/v3/core/v3.0.rst @@ -295,7 +295,7 @@ The following figure illustrates the first part of the terminology: *Codec* - The list of *codecs* specified for an array_ determine the encoded byte + The list of *codecs* specified for an array_ determines the encoded byte representation of each chunk in the store_. .. _metadata document: @@ -634,13 +634,12 @@ mandatory names: ``codecs`` ^^^^^^^^^^ - Specifies a list of codecs to be used for encoding and decoding chunks. The - value must be an array of objects, each object containing a member with - ``name`` whose value is a string referring to a v3 codec specification. The - codec object may also contain a ``configuration`` object which consists of - the parameter names and values as defined by the corresponding codec - specification. Since an ``array -> bytes`` codec must be specified, the - list cannot be empty. + Specifies a list of codecs to be used for encoding and decoding chunks. The + value MUST be an array of objects, where each object contains a member with the name + ``name`` whose value is a string. The ``name`` member identifies the specification for the codec. A + codec object MAY contain a member named ``configuration``, which is an object defined by the respective codec + specification. Because ``codecs`` MUST contain an ``array -> bytes`` codec, the + list cannot be empty (See :ref:`codecs `). The following members are optional: @@ -776,7 +775,7 @@ above, but using a (currently made up) extension data type:: - ``chunks`` has been replaced with ``chunk_grid``, - ``dimension_separator`` has been replaced with ``chunk_key_encoding``, - ``order`` has been replaced by the :ref:`transpose ` codec, - - the separate ``filters`` and ``compressor`` fields been combined into the single ``codecs`` field. + - the separate ``filters`` and ``compressor`` fields have been combined into the single ``codecs`` field. .. _group-metadata: @@ -1211,40 +1210,42 @@ the following procedure: 4. The chunk array ``A`` is equal to ``EC[0]``. +.. _codec-specification: + Specifying codecs ----------------- -To allow for flexibility to define and implement new codecs, this -specification does not define any codecs, nor restrict the set of -codecs that may be used. Each codec must be defined via a separate -specification. In order to refer to codecs in array metadata -documents, each codec must have a unique identifier, which is a URI -that dereferences to a human-readable specification of the codec. A -codec specification must declare the codec identifier, and describe -(or cite documents that describe) the encoding and decoding algorithms -and the format of the encoded data. - -A codec may have configuration parameters which modify the behaviour -of the codec in some way. For example, a compression codec may have a -compression level parameter, which is an integer that affects the -resulting compression ratio of the data. Configuration parameters must -be declared in the codec specification, including a definition of how -configuration parameters are represented as JSON. - -The Zarr core development team maintains a repository of codec -specifications, which are hosted alongside this specification in the -`zarr-specs GitHub repository`_, and which are -published on the `zarr-specs documentation Web site -`_. For ease of discovery, it is -recommended that codec specifications are contributed to the -zarr-specs GitHub repository. However, codec specifications may be -maintained by any group or organisation and published in any location -on the Web. For further details of the process for contributing a -codec specification to the zarr-specs GitHub repository, see -`ZEP 0 `_ which describes -the process for Zarr specification changes. - -Further details of how codecs are configured for an array are given in the `Array metadata`_ section. +.. _core-codecs: + +Core codecs +----------- + +This spec defines a set of codecs ("core codecs") which all Zarr implementations SHOULD implement in +order to ensure a minimal level of interoperability between Zarr implementations. +Each core codec is each defined by specification documents which are hosted in the +`zarr-specs GitHub repository`_, and are published on the `zarr-specs documentation web site +`_. The list of core codecs is part of the Zarr v3 specification. +Changes to the list of core codecs MUST be made via the same protocol used for +changing the Zarr v3 specification. Changes to the list of core codecs SHOULD be made +in close collaboration with extant Zarr v3 implementations. A new core codec SHOULD be added to the +list when a sufficient number of Zarr implementations support or intend to support that codec. +An existing core codec SHOULD be removed from the list when a sufficient number of implementation +developers and Zarr users deem the codec worth removing, e.g. because of a technical flaw in the +algorithm underlying the codec. + +Extension codecs +---------------- + +Zarr implementations MAY support a codec that is not in the list of core codecs +(hereafter termed an "extension codec"), provided the extension codec does not use an identifier +that is already used by a core codec; the identifiers of core codecs are reserved. + +This specification places no other constraints on extension codecs. It is possible, through discouraged, +that separate developers may define distinct codecs that use the same identifier. +To minimize the impact of such name collisions, codec developers are strongly encouraged +to publish their codec specifications as additions to the "extension codecs" section of Zarr v3 specification. +Publication in the "extension codecs" section does not confer primacy or an official designation to a codec. +The list of extension codecs exists expressly as a tool to enable coordinated codec development. Stores ======