diff --git a/content/en/blog/2023/http-conventions-declared-stable.md b/content/en/blog/2023/http-conventions-declared-stable.md new file mode 100644 index 000000000000..e3ec6a5bdc4e --- /dev/null +++ b/content/en/blog/2023/http-conventions-declared-stable.md @@ -0,0 +1,272 @@ +--- +title: HTTP semantic conventions declared stable +linkTitle: HTTP semconv are stable +date: 2023-11-06 +author: '[Trask Stalnaker](https://github.com/trask) (Microsoft)' +cSpell:ignore: chalin Liudmila Molkova +--- + +Early this year, we launched an effort to stabilize HTTP semantic conventions. +Today, we proudly announce that the HTTP semantic conventions are the _first_ +OpenTelemetry semantic conventions to be declared +**[stable](/docs/specs/otel/document-status/)**! This inaugural stable +[v1.23.0](https://github.com/open-telemetry/semantic-conventions/releases/tag/v1.23.0) +release marks a substantial advancement from earlier versions, featuring: + +- Enhancements resulting from + [convergence with the Elastic Common Schema](/blog/2023/ecs-otel-semconv-convergence/), + such as: + - The `url.*` namespace, which can be reused in the future by non-HTTP + semantic conventions + - Replacing the `net.peer.*` and `net.host.*` namespaces with `client.*` and + `server.*`, which + - Works better for logs, where there's no span kind to indicate which side + is peer and which side is host + - Simplifies correlation across client and server telemetry (e.g. since you + can join directly on `server.address` instead of joining where + `net.peer.addr` == `net.host.addr`) + - Provides a clean separation from the `network.*` namespace which is now + only for low-level network attributes + - More consistency around using the `http.request.*` and `http.response.*` + namespaces +- Improved consistency with Prometheus through standardization on seconds for + metric units. +- Streamlined attribute capture by omitting less useful attributes, reducing + telemetry capture, processing, and storage costs. +- Clarified the definition of default values, eliminating ambiguities when + attributes are absent. +- HTTP metrics are no longer vulnerable to cardinality explosion + - `http.request.method` is limited to a (configurable) set of known methods + - `server.address` and `server.port`, which are influenced by the `Host` + header, are now Opt-In on HTTP metrics + +## Migration plan + +Due to the significant number of modifications and the extensive user base +affected by them, we require existing HTTP instrumentations published by +OpenTelemetry to implement a migration plan that will assist users in +transitioning to stable HTTP semantic conventions. We intend to use a similar +strategy when stabilizing other semantic conventions. + +Specifically, when existing HTTP instrumentations published by OpenTelemetry are +updated to the stable HTTP semantic conventions, they: + +- Need to introduce an environment variable `OTEL_SEMCONV_STABILITY_OPT_IN` in + their existing major version, which accepts: + - `http` - emit the stable HTTP and networking conventions, and stop emitting + the old HTTP and networking conventions that the instrumentation emitted + previously. + - `http/dup` - emit both the old and the stable HTTP and networking + conventions, allowing for a phased rollout of the stable semantic + conventions. + - The default behavior (in the absence of one of these values) is to continue + emitting whatever version of the old HTTP and networking conventions the + instrumentation was emitting previously. +- Need to maintain (security patching at a minimum) their existing major version + for at least six months after it starts emitting both sets of conventions. +- May drop the environment variable in their next major version and emit only + the stable HTTP and networking conventions. + +## Summary of changes + +In this section, we summarize the changes made to the HTTP semantic conventions +from +[v1.20.0](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.20.0/specification/trace/semantic_conventions/http.md) +to +[v1.23.0 (stable)](https://github.com/open-telemetry/semantic-conventions/blob/v1.23.0/docs/http/). + +### Common attributes across HTTP client and server spans + + +| Change | Comments | +| --- | --- | +| `http.method` → `http.request.method` | Now captures only 9 common HTTP methods by default (configurable) plus `_OTHER` | +| `http.status_code` → `http.response.status_code` | | +| `http.request.header.` | • Dash (`"-"`) to underscore (`"_"`) normalization in `` has been removed
• On HTTP server spans: now must be provided to sampler | +| `http.response.header.` | Dash (`"-"`) to underscore (`"_"`) normalization in `` has been removed | +| `http.request_content_length` → `http.request.body.size` | • Recommended → Opt-In
• _Not marked stable yet_ | +| `http.response_content_length` → `http.response.body.size` | • Recommended → Opt-In
• _Not marked stable yet_ | +| `user_agent.original` | • On HTTP client spans: Recommended → Opt-In
• On HTTP server spans: now must be provided to sampler
• See note if [migrating from <= v1.18.0](#migrating-from--v1180) | +| `net.protocol.name` → `network.protocol.name` | Recommended → Conditionally required if not `http` and `network.protocol.version` is set | +| `net.protocol.version` → `network.protocol.version` | • Examples fixed: `2.0` → `2` and `3.0` → `3`
• See note if [migrating from <= v1.19.0](#migrating-from--v1190) | +| `net.sock.family` | Removed | +| `net.sock.peer.addr` → `network.peer.address` | On HTTP server spans: if `http.client_ip` was unknown, then also `net.sock.peer.addr` → `client.address`; `client.address` must be provided to sampler | +| `net.sock.peer.port` → `network.peer.port` | Now captured even if same as `server.port` | +| `net.sock.peer.name` | Removed | +| New: `http.request.method_original` | Only captured when `http.request.method` is `_OTHER` | +| New: `error.type` | | +{.td-initial .table .table-responsive} + + +References: + +- [Common attributes v1.20.0](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.20.0/specification/trace/semantic_conventions/http.md#common-attributes) +- [Common attributes v1.23.0 (stable)](https://github.com/open-telemetry/semantic-conventions/blob/v1.23.0/docs/http/http-spans.md#common-attributes) + +### HTTP client span attributes + + + + + +| Change | Comments | +| --- | --- | +| `http.url` → `url.full` | | +| `http.resend_count` → `http.request.resend_count` | | +| `net.peer.name` → `server.address` | | +| `net.peer.port` → `server.port` | Now captured even when same as default port for scheme | +{.td-initial .table .table-responsive .ot-table-first-row-60} + + +References: + +- [HTTP client span attributes v1.20.0](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.20.0/specification/trace/semantic_conventions/http.md#http-client) +- [HTTP client span attributes v1.23.0 (stable)](https://github.com/open-telemetry/semantic-conventions/blob/v1.23.0/docs/http/http-spans.md#http-client) + +### HTTP server span attributes + + +| Change | Comments | +| --- | --- | +| `http.route` | No change | +| `http.target` → `url.path` and `url.query` | Split into two separate attributes | +| `http.scheme` → `url.scheme` | Now factors in [X-Forwarded-Proto][], [Forwarded#proto][] headers | +| `http.client_ip` → `client.address` | If `http.client_ip` was unknown (i.e., no [X-Forwarded-For][], [Forwarded#for][] headers), then `net.sock.peer.addr` → `client.address`; now must be provided to sampler | +| `net.host.name` → `server.address` | Now based only on [Host][Host header], [:authority][HTTP/2 authority], [X-Forwarded-Host][], [Forwarded#host][] headers | +| `net.host.port` → `server.port` | Now based only on [Host][Host header], [:authority][HTTP/2 authority], [X-Forwarded-Host][X-Forwarded-Host], [Forwarded#host][] headers | +{.td-initial .table .table-responsive .ot-table-first-row-50} + + +References: + +- [HTTP server span attributes v1.20.0](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.20.0/specification/trace/semantic_conventions/http.md#http-server) +- [HTTP server span attributes v1.23.0 (stable)](https://github.com/open-telemetry/semantic-conventions/blob/v1.23.0/docs/http/http-spans.md#http-server) + +### HTTP client and server span names + +The `{http.method}` portion of span names is replace by `HTTP` when +`{http.method}` is `_OTHER`. + +See note if [migrating from `<= v1.17.0`](#migrating-from--v1170). + +References: + +- [HTTP client and server span names v1.20.0](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.20.0/specification/trace/semantic_conventions/http.md#name) +- [HTTP client and server span names v1.23.0 (stable)](https://github.com/open-telemetry/semantic-conventions/blob/v1.23.0/docs/http/http-spans.md#name) + +### HTTP client duration metric + +Metric changes: + +- **Name**: `http.client.duration` → `http.client.request.duration` +- **Unit**: `ms` → `s` +- **Description**: `Measures the duration of inbound HTTP requests.` → + `Duration of HTTP server requests.` +- **Histogram buckets**: boundaries updated to reflect change from milliseconds + to seconds, and zero bucket boundary removed +- **Attributes**: see table below + + +| Attribute change | Comments | +| --- | --- | +| `http.method` → `http.request.method` | Now captures only 9 common HTTP methods by default plus `_OTHER` | +| `http.status_code` → `http.response.status_code` | | +| `net.peer.name` → `server.address` | | +| `net.peer.port` → `server.port` | Now captured even when same as default port for scheme | +| `net.sock.peer.addr` | Removed | +| `net.protocol.name` → `network.protocol.name` | Recommended → Conditionally required if not `http` and `network.protocol.version` is set | +| `net.protocol.version` → `network.protocol.version` | Examples fixed: `2.0` → `2` and `3.0` → `3`; see note if [migrating from `<= v1.19.0`](#migrating-from--v1190) | +| New: `error.type` | | +{.td-initial .table .table-responsive} + + +References: + +- [Metric `http.client.duration` v1.20.0](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.20.0/specification/metrics/semantic_conventions/http-metrics.md#metric-httpclientduration) +- [Metric `http.client.request.duration` v1.23.0 (stable)](https://github.com/open-telemetry/semantic-conventions/blob/v1.23.0/docs/http/http-metrics.md#metric-httpclientrequestduration) + +### HTTP server duration metric + +Metric changes: + +- **Name**: `http.server.duration` → `http.server.request.duration` +- **Unit**: `ms` → `s` +- **Description**: `Measures the duration of inbound HTTP requests.` → + `Duration of HTTP server requests.` +- **Histogram buckets**: boundaries updated to reflect change from milliseconds + to seconds, and zero bucket boundary removed +- **Attributes**: see table below + + +| Attribute change | Comments | +| --- | --- | +| `http.route` | No change | +| `http.method` → `http.request.method` | Now captures only 9 common HTTP methods by default plus `_OTHER` | +| `http.status_code` → `http.response.status_code` | | +| `http.scheme` → `url.scheme` | Now factors in [`X-Forwarded-Proto` span][X-Forwarded-Proto], [`Forwarded#proto` span][Forwarded#proto] headers | +| `net.protocol.name` → `network.protocol.name` | Recommended → Conditionally required if not `http` and `network.protocol.version` is set | +| `net.protocol.version` → `network.protocol.version` | Examples fixed: `2.0` → `2` and `3.0` → `3`; see note if [migrating from `<= v1.19.0`](#migrating-from--v1190) | +| `net.host.name` → `server.address` | • Recommended → Opt-In (due to high-cardinality vulnerability since based on HTTP headers)
• Now based only on [`Host` span][Host header], [`:authority` span][HTTP/2 authority], [`X-Forwarded-Host` span][X-Forwarded-Host], [`Forwarded#host` span][Forwarded#host] headers | +| `net.host.port` → `server.port` | • Recommended → Opt-In (due to high-cardinality vulnerability since based on HTTP headers)
• Now based only on [`Host` span][Host header], [`:authority` span][HTTP/2 authority], [`X-Forwarded-Host` span][X-Forwarded-Host], [`Forwarded#host` span][Forwarded#host] headers | +| New: `error.type` | | +{.td-initial .table .table-responsive .ot-table-first-row-50} + + +References: + +- [Metric `http.server.duration` v1.20.0](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.20.0/specification/metrics/semantic_conventions/http-metrics.md#metric-httpserverduration) +- [Metric `http.server.request.duration` v1.23.0 (stable)](https://github.com/open-telemetry/semantic-conventions/blob/v1.22.0/docs/http/http-metrics.md#metric-httpserverrequestduration) + +## Migrating from a version earlier than v1.20.0? + +### Migrating from `<= v1.19.0` + +- `http.flavor` → `network.protocol.version` + - Examples fixed: `2.0` → `2` and `3.0` → `3` + +### Migrating from `<= v1.18.0` + +- `http.user_agent` → `user_agent.original` + +### Migrating from `<= v1.17.0` + +#### HTTP server span name + +- When `http.route` is available:
`{http.route}` → + `{summary} {http.route}` +- When `http.route` is not available:
`HTTP {http.method}` → + `{summary}` + +Where `{summary}` is `{http.method}`, unless `{http.method}` is `_OTHER`, in +which case `{summary}` is `HTTP`. + +#### HTTP client span name + +- `HTTP {http.method}` → `{summary}` + +Where `{summary}` is `{http.method}`, unless `{http.method}` is `_OTHER`, in +which case `{summary}` is `HTTP`. + +[Host header]: https://tools.ietf.org/html/rfc7230#section-5.4 +[HTTP/2 authority]: https://tools.ietf.org/html/rfc9113#section-8.3.1 +[Forwarded#for]: + https://developer.mozilla.org/docs/Web/HTTP/Headers/Forwarded#for +[Forwarded#proto]: + https://developer.mozilla.org/docs/Web/HTTP/Headers/Forwarded#proto +[Forwarded#host]: + https://developer.mozilla.org/docs/Web/HTTP/Headers/Forwarded#host +[X-Forwarded-For]: + https://developer.mozilla.org/docs/Web/HTTP/Headers/X-Forwarded-For +[X-Forwarded-Proto]: + https://developer.mozilla.org/docs/Web/HTTP/Headers/X-Forwarded-Proto +[X-Forwarded-Host]: + https://developer.mozilla.org/docs/Web/HTTP/Headers/X-Forwarded-Host + +## Community Kudos + +This was a massive community effort: thanks to all who got involved! A special +thanks to [Liudmila Molkova](https://github.com/lmolkova) for sharing her HTTP +domain expertise, which helped drive this effort every step of the way. diff --git a/static/refcache.json b/static/refcache.json index 3f61fb2a28a2..f64a098baacf 100644 --- a/static/refcache.json +++ b/static/refcache.json @@ -619,6 +619,30 @@ "StatusCode": 206, "LastSeen": "2023-06-29T18:51:23.515285-04:00" }, + "https://developer.mozilla.org/docs/Web/HTTP/Headers/Forwarded#for": { + "StatusCode": 206, + "LastSeen": "2023-11-03T03:04:16.900109-04:00" + }, + "https://developer.mozilla.org/docs/Web/HTTP/Headers/Forwarded#host": { + "StatusCode": 206, + "LastSeen": "2023-11-03T03:04:48.340922-04:00" + }, + "https://developer.mozilla.org/docs/Web/HTTP/Headers/Forwarded#proto": { + "StatusCode": 206, + "LastSeen": "2023-11-03T03:04:27.385988-04:00" + }, + "https://developer.mozilla.org/docs/Web/HTTP/Headers/X-Forwarded-For": { + "StatusCode": 206, + "LastSeen": "2023-11-03T03:04:11.582225-04:00" + }, + "https://developer.mozilla.org/docs/Web/HTTP/Headers/X-Forwarded-Host": { + "StatusCode": 206, + "LastSeen": "2023-11-03T03:04:43.193675-04:00" + }, + "https://developer.mozilla.org/docs/Web/HTTP/Headers/X-Forwarded-Proto": { + "StatusCode": 206, + "LastSeen": "2023-11-03T03:04:22.148405-04:00" + }, "https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS": { "StatusCode": 206, "LastSeen": "2023-06-29T15:53:59.660121-04:00" @@ -2263,6 +2287,10 @@ "StatusCode": 200, "LastSeen": "2023-06-30T08:32:10.766549-04:00" }, + "https://github.com/lmolkova": { + "StatusCode": 200, + "LastSeen": "2023-11-03T03:04:54.119611-04:00" + }, "https://github.com/mahboubii/grpcmetrics": { "StatusCode": 200, "LastSeen": "2023-06-30T09:15:29.922451-04:00" @@ -3487,6 +3515,10 @@ "StatusCode": 200, "LastSeen": "2023-11-01T21:52:00.884558-04:00" }, + "https://github.com/open-telemetry/semantic-conventions/releases/tag/v1.23.0": { + "StatusCode": 200, + "LastSeen": "2023-11-03T20:22:42.689401141Z" + }, "https://github.com/opentracing": { "StatusCode": 200, "LastSeen": "2023-06-30T08:40:39.909083-04:00" @@ -5331,6 +5363,10 @@ "StatusCode": 200, "LastSeen": "2023-06-29T18:42:20.913106-04:00" }, + "https://tools.ietf.org/html/rfc9113#section-8.3.1": { + "StatusCode": 200, + "LastSeen": "2023-11-03T03:04:37.939785-04:00" + }, "https://tracetest.io/": { "StatusCode": 206, "LastSeen": "2023-08-14T12:06:16.100357-03:00"