From f41061e560e02292225103628faa3ffeb966bec5 Mon Sep 17 00:00:00 2001 From: Doug Davis Date: Thu, 19 Sep 2019 12:41:15 -0400 Subject: [PATCH] make extensions follow the same pattern as normal attrs (#505) * issue 498 - require extensions to have ce- version Signed-off-by: Doug Davis * fix mqtt Signed-off-by: Doug Davis * add pointer for mark Signed-off-by: Doug Davis --- amqp-format.md | 8 +++--- amqp-transport-binding.md | 7 +++--- avro-format.md | 5 ++-- documented-extensions.md | 2 +- extensions/distributed-tracing.md | 10 ++++++++ http-transport-binding.md | 41 ++++++++++++------------------- json-format.md | 7 +++--- mqtt-transport-binding.md | 38 ++++++++++++++-------------- spec.md | 26 +++++++++++++++++--- 9 files changed, 85 insertions(+), 59 deletions(-) diff --git a/amqp-format.md b/amqp-format.md index 4f2a21eb5..fb841537b 100644 --- a/amqp-format.md +++ b/amqp-format.md @@ -76,15 +76,17 @@ representation before surfacing the value to the API. An AMQP implementation SHOULD convert from/to the native runtime or language type system to the AMQP type system directly without translating through strings whenever possible. -Extension specifications MAY define diverging mapping rules for the values of -attributes they define. +Extension specifications MAY define a +[secondary mapping rule](./amqp-transport-binding.md#313-metadata-headers) +for the values of attributes they define, but MUST also include the previously +defined primary mapping. For instance, the attribute value may be a data structure defined in a standard outside of CloudEvents, with a formal AMQP mapping, and there might be risk of translation errors or information loss when the original format is not preserved. -An extension specification that defines a diverging mapping rule for AMQP, and +An extension specification that defines a secondary mapping rule for AMQP, and any revision of such a specification, MUST also define explicit mapping rules for all other event formats that are part of the CloudEvents core at the time of the submission or revision. diff --git a/amqp-transport-binding.md b/amqp-transport-binding.md index 5867e1822..3159cd4e0 100644 --- a/amqp-transport-binding.md +++ b/amqp-transport-binding.md @@ -160,13 +160,14 @@ All [CloudEvents][ce] attributes with exception of `datacontenttype` and `data` MUST be individually mapped to and from the AMQP [application-properties][app-properties] section, with exceptions noted below. -CloudEvents extensions that define their own attributes MAY define a diverging +CloudEvents extensions that define their own attributes MAY define a secondary mapping to AMQP properties for those attributes, also in different message sections, especially if specific attributes or their names need to align with AMQP features or with other specifications that have explicit AMQP header -bindings. +bindings. However, they MUST also include the previously defined primary +mapping. -An extension specification that defines a diverging mapping rule for AMQP, and +An extension specification that defines a secondary mapping rule for AMQP, and any revision of such a specification, MUST also define explicit mapping rules for all other transport bindings that are part of the CloudEvents core at the time of the submission or revision. diff --git a/avro-format.md b/avro-format.md index 0c1435138..a87445ce4 100644 --- a/avro-format.md +++ b/avro-format.md @@ -56,8 +56,9 @@ The CloudEvents type system MUST be mapped to Avro types as follows. | URI-reference | [string][avro-primitives] following [RFC 3986 ยง4.1][rfc3986-section41] | | Timestamp | [string][avro-primitives] following [RFC 3339][rfc3339] (ISO 8601) | -Extension specifications MAY define diverging mapping rules for the values of -attributes they define. +Extension specifications MAY define secondary mapping rules for the values of +attributes they define, but MUST also include the previously defined primary +mapping. ### 2.3 OPTIONAL Attributes diff --git a/documented-extensions.md b/documented-extensions.md index 39a3c47d6..9096e548b 100644 --- a/documented-extensions.md +++ b/documented-extensions.md @@ -35,7 +35,7 @@ Extensions always follow a common placement strategy for in-memory formats (e.g. representations. Transport bindings (e.g. [HTTP](http-transport-binding.md), [MQTT](mqtt-transport-binding.md), [AMPQ](amqp-transport-binding.md), [NATS](nats-transport-binding.md)) provide default placement for extensions, but -an extension MAY require special representation when transported (e.g. tracing +an extension MAY require special secondary representation when transported (e.g. tracing standards that require specific headers). Extension authors SHOULD only require special representation in transport bindings where extensions integrate with pre-existing specs; extensions with custom transport bindings are much more diff --git a/extensions/distributed-tracing.md b/extensions/distributed-tracing.md index 1b78ee553..44a627497 100644 --- a/extensions/distributed-tracing.md +++ b/extensions/distributed-tracing.md @@ -43,3 +43,13 @@ CURL -X POST example/webhook.json \ -H 'content-type: application/cloudevents+json' \ -d '@sample-event.json' ``` + +## Conflicts + +Since this extension defines secondary, special, seialization that differs +from other CloudEvents attributes, it is possible that the values of these two +could differ by the time the event is received at a destination. In those +cases, the serialization that followed the "general CloudEvents serialization +rules" MUST be used as the CloudEvents attribute. The other, secodary, +mapping MAY be picked-up and offered to the receiving application as +"additional" metadata. diff --git a/http-transport-binding.md b/http-transport-binding.md index 03cdcec24..59e3825ad 100644 --- a/http-transport-binding.md +++ b/http-transport-binding.md @@ -70,8 +70,9 @@ _binary_, _structured_ and _batched_. Every compliant implementation SHOULD support the _structured_ and _binary_ modes. In the _binary_ content mode, the value of the event `data` is placed into the -HTTP request or response body as-is, with the `datacontenttype` attribute value -declaring its media type; all other event attributes are mapped to HTTP headers. +HTTP request, or response, body as-is, with the `datacontenttype` attribute +value declaring its media type in the HTTP `Content-Type` header; all other +event attributes are mapped to HTTP headers. In the _structured_ content mode, event metadata attributes and event data are placed into the HTTP request or response body using an @@ -102,15 +103,9 @@ identically to [HTTP over TLS]([RFC2818][RFC2818]). This specification does not further define any of the core [CloudEvents][ce] event attributes. -Two of the event attributes, `datacontenttype` and `data` are handled specially -and mapped onto HTTP constructs, all other attributes are transferred as -metadata without further interpretation (except that extensions MAY define -additional header mappings). - This mapping is intentionally robust against changes, including the addition and removal of event attributes, and also accommodates vendor extensions to the -event metadata. Any mention of event attributes other than `datacontenttype` and -`data` is exemplary. +event metadata. ### 2.1. datacontenttype Attribute @@ -161,7 +156,8 @@ efficient transfer and without transcoding effort. For the _binary_ mode, the HTTP `Content-Type` header value corresponds to (MUST be populated from or written to) the CloudEvents `datacontenttype` -attribute. +attribute. Note that a `ce-datacontenttype` HTTP header MUST NOT also be +present in the message. #### 3.1.2. Event Data Encoding @@ -169,26 +165,21 @@ The [`data`](#22-data) byte-sequence is used as the HTTP message body. #### 3.1.3. Metadata Headers -All [CloudEvents][ce] attributes with the exception of `datacontenttype` MUST be -individually mapped to and from distinct HTTP message headers, with exceptions -noted below. +All other [CloudEvents][ce] attributes, including extensions, MUST be +individually mapped to and from distinct HTTP message header. -CloudEvents extensions that define their own attributes MAY define a diverging +CloudEvents extensions that define their own attributes MAY define a secondary mapping to HTTP headers for those attributes, especially if specific attributes need to align with HTTP features or with other specifications that have explicit -HTTP header bindings. - -An extension specification that defines a diverging mapping rule for HTTP, and -any revision of such a specification, MUST also define explicit mapping rules -for all other transport bindings that are part of the CloudEvents core at the -time of the submission or revision. +HTTP header bindings. Note that these attributes MUST also still appear in the +HTTP message as HTTP headers with the `ce-` prefix as noted in +[HTTP Header Names](#3131-http-header-names). ##### 3.1.3.1. HTTP Header Names -Except for attributes -[explicitly handled in this specification](#2-use-of-cloudevents-attributes), -the naming convention for the HTTP header mapping of well-known CloudEvents -attributes is that each attribute name MUST be prefixed with "ce-". +Except where noted, all CloudEvents context attributes, including extensions, +MUST be mapped to HTTP headers with the same name as the attribute name but +prefixed with `ce-`. Examples: @@ -213,7 +204,7 @@ String values MUST be percent-encoded as described in [RFC3986, section 2.4][rfc3986-section-2-4] before applying the header encoding rules described in [RFC7230, section 3.2.6][rfc7230-section-3-2s6]. -When decoding an HTTP message into a Cloud Event, these rules MUST be applied in +When decoding an HTTP message into a CloudEvent, these rules MUST be applied in reverse -- [RFC7230, section 3.2.6][rfc7230-section-3-2-6] decoding to an ASCII string, and then a **single round** of percent-decoding as described in [RFC3986, section 2.4][rfc3986-section-2-4] to produce a valid UTF-8 String. diff --git a/json-format.md b/json-format.md index ab7d4d610..f1fbfdd09 100644 --- a/json-format.md +++ b/json-format.md @@ -70,15 +70,16 @@ with exceptions noted below. | URI-reference | [string][json-string] following [RFC 3986][rfc3986] | | Timestamp | [string][json-string] following [RFC 3339][rfc3339] (ISO 8601) | -Extension specifications MAY define diverging mapping rules for the values of -attributes they define. +Extension specifications MAY define secondary mapping rules for the values of +attributes they define, but MUST also include the previously defined primary +mapping. For instance, the attribute value might be a data structure defined in a standard outside of CloudEvents, with a formal JSON mapping, and there might be risk of translation errors or information loss when the original format is not preserved. -An extension specification that defines a diverging mapping rule for JSON, and +An extension specification that defines a secondary mapping rule for JSON, and any revision of such a specification, MUST also define explicit mapping rules for all other event formats that are part of the CloudEvents core at the time of the submission or revision. diff --git a/mqtt-transport-binding.md b/mqtt-transport-binding.md index 1a2227465..f69bd13f2 100644 --- a/mqtt-transport-binding.md +++ b/mqtt-transport-binding.md @@ -65,9 +65,9 @@ placed into the MQTT PUBLISH message payload section using an In the _binary_ content mode, the value of the event `data` is placed into the MQTT PUBLISH message's payload section as-is, with the -`datacontenttype` attribute value declaring its media type; all other event -attributes are mapped to the MQTT PUBLISH message's [properties -section][5-publish-properties]. +`datacontenttype` attribute value declaring its media type in the MQTT +PUBLISH message's [`Content Type`][5-content-type] property; all other +event attributes are mapped to User Property fields. ### 1.4. Event Formats @@ -88,14 +88,9 @@ mandate specific existing features to be used. This specification does not further define any of the [CloudEvents][ce] event attributes. -Two of the event attributes, `datacontenttype` and `data` are handled specially -and mapped onto MQTT constructs, all other attributes are transferred as -metadata without further interpretation. - This mapping is intentionally robust against changes, including the addition and removal of event attributes, and also accommodates vendor extensions to the -event metadata. Any mention of event attributes other than `datacontenttype` and -`data` is exemplary. +event metadata. ### 2.1. datacontenttype Attribute @@ -154,19 +149,15 @@ payload of the MQTT PUBLISH message. #### 3.1.3. Metadata Headers -All [CloudEvents][ce] attributes with exception of `datacontenttype` and `data` -MUST be individually mapped to and from the User Property fields in the MQTT -PUBLISH message, with exceptions noted below. +All other [CloudEvents][ce] context attributes, including extensions, MUST be +individually mapped to and from the User Property fields in the MQTT +PUBLISH message. -CloudEvents extensions that define their own attributes MAY define a diverging +CloudEvents extensions that define their own attributes MAY define a secondary mapping to MQTT user properties or features for those attributes, especially if specific attributes need to align with MQTT features, or with other -specifications that have explicit MQTT header bindings. - -An extension specification that defines a diverging mapping rule for MQTT, and -any revision of such a specification, MUST also define explicit mapping rules -for all other transport bindings that are part of the CloudEvents core at the -time of the submission or revision. +specifications that have explicit MQTT header bindings. However, they MUST +also include the previously defined primary mapping. ##### 3.1.3.1 User Property Names @@ -203,6 +194,7 @@ type: com.example.someevent time: 2018-04-05T03:56:24Z id: 1234-1234-1234 source: /mycontext/subcontext +datacontenttype: application/json; charset=utf-8 .... further attributes ... ------------------ payload ------------------- @@ -262,6 +254,10 @@ Content Type: application/cloudevents+json; charset=utf-8 { "specversion" : "0.4-wip", "type" : "com.example.someevent", + "time" : 2018-04-05T03:56;24Z, + "id" : 1234-1234-1234, + "source" : "/mycontext/subcontext", + "datacontenttype" : "application/json; charset=utf-8", ... further attributes omitted ... @@ -287,6 +283,10 @@ Topic Name: mytopic { "specversion" : "0.4-wip", "type" : "com.example.someevent", + "time" : 2018-04-05T03:56;24Z, + "id" : 1234-1234-1234, + "source" : "/mycontext/subcontext", + "datacontenttype" : "application/json; charset=utf-8", ... further attributes omitted ... diff --git a/spec.md b/spec.md index 0e21a931a..789b0fc95 100644 --- a/spec.md +++ b/spec.md @@ -406,8 +406,10 @@ See [CloudEvent Attributes Extensions](primer.md#cloudevent-attribute-extensions) for additional information concerning the use and definition of extensions. -This specification places no restriction on the type or semantics of the -extension attributes. Each definition of an extensions SHOULD fully define all +This specification places no restriction on the semantics of the +extension attributes, but they MUST use one of the types defined within the +[type system](#type-system). +Each definition of an extensions SHOULD fully define all aspects of the attribute - e.g. its name, semantic meaning and possible values or even to indicate that it places no restrictions on its values. New extension definitions SHOULD use a name that is descriptive enough to reduce the chances @@ -419,13 +421,31 @@ extensions that might be of interest. Each specification that defines how to serialize a CloudEvent will define how extension attributes will appear. +Extension attribtue MUST be serialized using the same general pattern as all +CloudEvents context attributes. For example, in binary HTTP, that means they +MUST appear as HTTP headers with the `ce-` prefix. The specification of an +attribute MAY define a secondary serialization where the data is duplicated +in some other location within the message. + +In cases where a secondary serialization is defined, the extension +specification MUST also state what a receiver of the CloudEvent is to do +if the data differs between those two serialization locations. Additionally, +senders need to be prepared for intermediaries, and receivers, to not +know about their extension and therefore the specialized serialization version +version will most likely not be processed as a CloudEvent extension attribute. + +Many transports support the ability for senders to include additonal metadata, +for example as HTTP headers. While a CloudEvents receiver is not mandated to +process and pass them along, it is RECOMMENDED that they do so via some +mechanism that makes it clear they are non-CloudEvents metadata. + Here is an example that illustrates the need for additional attributes. In many IoT and enterprise use cases, an event could be used in a serverless application that performs actions across multiple types of events. To support such use cases, the event producer will need to add additional identity attributes to the "context attributes" which the event consumers can use to correlate this event with the other events. If such identity attributes happen to be part of the -event "data", the event producer SHOULD also add the identity attributes to the +event "data", the event producer would also add the identity attributes to the "context attributes" so that event consumers can easily access this information without needing to decode and examine the event data. Such identity attributes can also be used to help intermediate gateways determine how to route the