Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: update observability/telemetry overview pages #6563

Merged
merged 38 commits into from
Feb 7, 2025
Merged
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
be81ad3
DOC-293: Restructure Telemetry overview
Meschreiber Jan 16, 2025
5308360
Restructure observability overview
Meschreiber Jan 16, 2025
23d6b58
Copyedits
Meschreiber Jan 16, 2025
7df3a1d
Callout Odyssey course
Meschreiber Jan 16, 2025
4e5e62d
Merge branch 'dev' into docs/ms/update-telemetry-overview
Meschreiber Jan 22, 2025
9e0b04c
Add mermaid diagrams and example configs
Meschreiber Jan 22, 2025
b835a64
Restructure
Meschreiber Jan 22, 2025
4d9a97a
Add pared down client debugging page
Meschreiber Jan 22, 2025
13cc66c
Fix yaml formatting
Meschreiber Jan 22, 2025
61b3a95
Add telemetry context
Meschreiber Jan 23, 2025
51cc6f0
Update client o11y example configs
Meschreiber Jan 23, 2025
06da027
Update telemetry example configs
Meschreiber Jan 23, 2025
b5e06f8
Apply suggestions from code review
Meschreiber Jan 24, 2025
70db821
Merge branch 'dev' into docs/ms/update-telemetry-overview
Meschreiber Jan 24, 2025
3267580
Apply suggestions from code review
Meschreiber Jan 24, 2025
b125633
Apply suggestions from code review
Meschreiber Jan 24, 2025
7b7206f
Remove links from table
Meschreiber Jan 24, 2025
2a21cda
Merge branch 'dev' into docs/ms/update-telemetry-overview
Meschreiber Jan 30, 2025
e82e843
Merge branch 'dev' into docs/ms/update-telemetry-overview
Meschreiber Jan 30, 2025
849eb1b
Merge branch 'dev' into docs/ms/update-telemetry-overview
Meschreiber Jan 30, 2025
7a2f12d
Merge branch 'dev' into docs/ms/update-telemetry-overview
Meschreiber Jan 30, 2025
b821fe1
Merge branch 'dev' into docs/ms/update-telemetry-overview
Meschreiber Jan 30, 2025
50519c4
Merge branch 'dev' into docs/ms/update-telemetry-overview
smyrick Jan 31, 2025
406617b
Typo
Meschreiber Feb 4, 2025
cc6eb4c
Suggestions from review
Meschreiber Feb 4, 2025
1fe0095
Merge branch 'dev' into docs/ms/update-telemetry-overview
Meschreiber Feb 4, 2025
137c4d1
Add additional router configs
Meschreiber Feb 5, 2025
8491c46
Merge branch 'dev' into docs/ms/update-telemetry-overview
Meschreiber Feb 5, 2025
9069383
Fix headers
Meschreiber Feb 5, 2025
759b259
Copyedit
Meschreiber Feb 5, 2025
a6eda60
Correct request logging section
Meschreiber Feb 5, 2025
3a2da88
Update logging requests
Meschreiber Feb 5, 2025
b0534fd
Typo
Meschreiber Feb 5, 2025
3383a94
Merge branch 'dev' into docs/ms/update-telemetry-overview
Meschreiber Feb 6, 2025
0eafaed
Add contional aspect back to request logging
Meschreiber Feb 7, 2025
5210514
Remove samples
Meschreiber Feb 7, 2025
5fbf89f
Add back samples
Meschreiber Feb 7, 2025
cc878c6
Remove errors and timeouts samples
Meschreiber Feb 7, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
title: Conditions
subtitle: Set conditions for when events or instruments are triggered
description: Set conditions for when events or instruments are triggered in the Apollo GraphOS Router.
context:
- telemetry
---

You can set conditions for when an [instrument](/router/configuration/telemetry/instrumentation/instruments) should be mutated or an [event](/router/configuration/telemetry/instrumentation/events) should be triggered.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
title: Events
subtitle: Capture events from the router's request lifecycle
description: Capture standard and custom events from the Apollo GraphOS Router's request lifecycle services.
context:
- telemetry
---

import RouterServices from '../../../../../shared/router-lifecycle-services.mdx';
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
title: Instruments
subtitle: Collect measurements with standard and custom instruments
description: Create and customize instruments to collect data and report measurements from the Apollo GraphOS Router's request lifecycle services.
context:
- telemetry
---

import RouterServices from '../../../../../shared/router-lifecycle-services.mdx';
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
title: Selectors
subtitle: Select data from the router pipeline to extract
description: Extract and select data from the Apollo GraphOS Router's pipeline services to attach to telemetry.
context:
- telemetry
---
import RouterServices from '../../../../../shared/router-lifecycle-services.mdx';

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
title: Spans
subtitle: Add router lifecycle context to traces
description: Use spans to add contextual information from the Apollo GraphOS Router or Apollo Router Core to traces displayed by your application performance monitors (APM).
context:
- telemetry
---

import RouterServices from '../../../../../shared/router-lifecycle-services.mdx';
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
title: OpenTelemetry standard attributes
subtitle: Attach standard attributes to router telemetry
description: Attach OpenTelemetry (OTel) standard attributes to Apollo GraphOS Router or Apollo Router Core telemetry.
context:
- telemetry
---

import RouterServices from '../../../../../shared/router-lifecycle-services.mdx';
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
title: Router Instruments
subtitle: Standard metric instruments for the router's request lifecycle
description: Reference of standard metric instruments for the request lifecycle of GraphOS Router and Apollo Router Core. Consumable via the router's metrics exporters.
context:
- telemetry
---

## Standard metric instruments
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
title: Router Logging
subtitle: Configure logging in the router
description: Configure logging in the Apollo GraphOS Router or Apollo Router Core. Set the log level and output format.
context:
- telemetry
---

GraphOS Router and Apollo Router Core provide built-in logging to capture records about their activity.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
title: Router Logging to stdout
subtitle: Configure logging to stdout
description: Configure logging output to stdout in the Apollo GraphOS Router or Apollo Router Core. Format in human-readable text or machine-readable JSON.
context:
- telemetry
---

You can configure GraphOS Router or Apollo Router Core logging to be directed to stdout, and its output format can be set to text or JSON.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
title: Datadog exporter (via OTLP)
subtitle: Configure the Datadog exporter for metrics
description: Configure the Datadog exporter for metrics via OpenTelemetry Protocol (OTLP) in the Apollo GraphOS Router or Apollo Router Core.
context:
- telemetry
---

Enable and configure the [OTLP exporter](/router/configuration/telemetry/exporters/metrics/otlp) for metrics in the GraphOS Router or Apollo Router Core for use with [Datadog](https://www.datadoghq.com/).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
title: Dynatrace exporter (via OTLP)
subtitle: Configure the Dynatrace exporter for metrics
description: Configure the Dynatrace exporter for metrics via OpenTelemetry Protocol (OTLP) in the Apollo Router.
context:
- telemetry
---

Enable and configure the [OTLP exporter](/router/configuration/telemetry/exporters/metrics/otlp) for metrics in the Apollo Router for use with [Dynatrace](https://dynatrace.com/).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
title: New Relic exporter (via OTLP)
subtitle: Configure the New Relic exporter for metrics
description: Configure the New Relic exporter for metrics via OpenTelemetry Protocol (OTLP) in the Apollo GraphOS Router or Apollo Router Core.
context:
- telemetry
---

Enable and configure the [OTLP exporter](/router/configuration/telemetry/exporters/metrics/otlp) for metrics in the GraphOS Router or Apollo Router Core for use with [New Relic](https://newrelic.com/).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
title: OpenTelemetry Protocol (OTLP) exporter
subtitle: Configure the OpenTelemetry Protocol (OTLP) exporter for metrics
description: Configure the OpenTelemetry Protocol (OTLP) exporter for metrics in the Apollo GraphOS Router or Apollo Router Core.
context:
- telemetry
---
import BatchProcessorPreamble from '../../../../../shared/batch-processor-preamble.mdx';
import BatchProcessorRef from '../../../../../shared/batch-processor-ref.mdx';
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ subtitle: Export router metrics
description: Collect and export metrics from the Apollo GraphOS Router or Apollo Router Core for Prometheus, OpenTelemetry Protocol (OTLP), Datadog, and New Relic.
redirectFrom:
- /technotes/TN0015-router-to-apm-via-opentelemetry/
context:
- telemetry
---

The GraphOS Router and Apollo Router Core support collection of metrics with [OpenTelemetry](https://opentelemetry.io/), with exporters for:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
title: Prometheus exporter
subtitle: Configure the Prometheus metrics exporter
description: Configure the Prometheus metrics exporter endpoint in the Apollo GraphOS Router or Apollo Router Core.
context:
- telemetry
---

Enable and configure the [Prometheus](https://www.prometheus.io/) exporter for metrics in the GraphOS Router or Apollo Router Core.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
title: Datadog exporter (via OTLP)
subtitle: Configure the Datadog exporter for tracing
description: Configure the Datadog exporter for tracing via OpenTelemetry Protocol (OTLP) in the Apollo GraphOS Router or Apollo Router Core.
context:
- telemetry
---
import BatchProcessorPreamble from '../../../../../shared/batch-processor-preamble.mdx';
import BatchProcessorRef from '../../../../../shared/batch-processor-ref.mdx';
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
title: Dynatrace exporter (via OTLP)
subtitle: Configure the Dynatrace exporter for tracing
description: Configure the Dynatrace exporter for tracing via OpenTelemetry Protocol (OTLP) in the Apollo Router.
context:
- telemetry
---

Enable and configure the [OTLP exporter](/router/configuration/telemetry/exporters/tracing/otlp) for tracing in the Apollo Router for use with [Dynatrace](https://dynatrace.com/).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
title: Jaeger exporter (via OTLP)
subtitle: Configure the Jaeger exporter for tracing
description: Configure the Jaeger exporter for tracing via OpenTelemetry Protocol (OTLP) in the Apollo GraphOS Router or Apollo Router Core.
context:
- telemetry
---

import BatchProcessorPreamble from "../../../../../shared/batch-processor-preamble.mdx";
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
title: New Relic exporter (via OTLP)
subtitle: Configure the New Relic exporter for tracing
description: Configure the New Relic exporter for tracing via OpenTelemetry Protocol (OTLP) in the Apollo GraphOS Router or Apollo Router Core.
context:
- telemetry
---

Enable and configure the [OTLP exporter](/router/configuration/telemetry/exporters/tracing/otlp) for tracing in the GraphOS Router or Apollo Router Core for use with [New Relic](https://newrelic.com/).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
title: OpenTelemetry Protocol (OTLP) exporter
subtitle: Configure the OpenTelemetry Protocol exporter for tracing
description: Configure the OpenTelemetry Protocol (OTLP) exporter for tracing in the Apollo GraphOS Router or Apollo Router Core.
context:
- telemetry
---
import BatchProcessorPreamble from '../../../../../shared/batch-processor-preamble.mdx';
import BatchProcessorRef from '../../../../../shared/batch-processor-ref.mdx';
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
title: Router Tracing
subtitle: Collect tracing information from the router
description: Collect and export tracing information from the Apollo GraphOS Router or Apollo Router Core. Supports OpenTelemetry Protocol (OTLP), Datadog, New Relic, Jaeger, Zipkin.
context:
- telemetry
---

The GraphOS Router and Apollo Router Core support collection of traces with [OpenTelemetry](https://opentelemetry.io/), with exporters for:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
title: Zipkin exporter
subtitle: Configure the Zipkin exporter for tracing
description: Enable and configure the Zipkin exporter for tracing in the Apollo GraphOS Router or Apollo Router Core.
context:
- telemetry
---
import BatchProcessorPreamble from '../../../../../shared/batch-processor-preamble.mdx';
import BatchProcessorRef from '../../../../../shared/batch-processor-ref.mdx';
Expand Down
100 changes: 100 additions & 0 deletions docs/source/routing/observability/debugging-client-requests.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
---
title: Debugging Client Requests to GraphOS Router
subtitle: Options for analyzing and debugging incoming requests
description: Learn how to use GraphOS router telemetry and GraphOS Insights to inspect and debug incoming HTTP client requests.
context:
- telemetry
---

By default, the GraphOS Router operates without generating HTTP request logs or exporting telemetry metrics beyond what it sends to GraphOS. This minimizes potentially high observability costs that can result from high request volumes.

If you need more data than what GraphOS provides by default, you can configure your router to collect and export additional telemetry.

## Using GraphOS Insights

GraphOS Studio lets you analyze data from failed requests, such as GraphQL error messages ([if enabled](/graphos/routing/graphos-reporting#errors)) and the ID of the client making the request. You can also [segment your insights data](/graphos/platform/insights/client-segmentation) based on the client ID.

<Tip>

[Learn how to ensure client IDs are included in all requests.](/graphos/routing/observability/client-id-enforcement)

</Tip>

## Enabling additional telemetry

Use the methods below if you need information outside of what's presented in GraphOS Studio to debug client requests.

<Note>

If you want to debug client requests in your own environment, Apollo recommends first doing so in a non-production environment or using logic to debug on a per-request basis.

</Note>

### Telemetry spans and traces

Once you enable [router telemetry](/graphos/routing/observability/telemetry), traces can have spans labeled `router`. These spans contain the HTTP information for a given client request via [OpenTelemetry standard attributes](/graphos/reference/router/telemetry/instrumentation/standard-attributes#router) such as `error.type` and `http.response.status_code`.

```yaml title="router.yaml"
Meschreiber marked this conversation as resolved.
Show resolved Hide resolved
telemetry:
instrumentation:
spans:
router:
attributes:
http.response.status_code: true
error.type: true
```

These attributes include helpful information about the request and response. Spans, however, don't include an attribute for the HTTP body text. If you want to see the GraphQL operation, first check your [GraphOS Insights](/graphos/platform/insights) or use the other options listed below.

### Request and response logging

You can include the request body in your telemetry by instrumenting [`events` in the supergraph service](/graphos/reference/router/telemetry/instrumentation/events#event-configuration) like so:

```yaml title="router.yaml"
Meschreiber marked this conversation as resolved.
Show resolved Hide resolved
telemetry:
instrumentation:
spans:
default_attribute_requirement_level: recommended
mode: spec_compliant
events:
supergraph:
"req":
message: "logged request"
level: info
on: "request"
attributes:
graphql.document: true
```

### Debugging router logs

By default, the router uses the `info` level for its logging. [Enabling other logging levels](/graphos/reference/router/telemetry/log-exporters/overview) can help debug specific scenarios. Using non-`info` level configurations is only recommended for local or non-production environments.

### Rhai scripts and coprocessors

Hooking into the router service layer with either [Rhai scripts](/graphos/routing/customization/rhai) or [coprocessors](/graphos/routing/customization/coprocessor) gives you access to the full HTTP request before processing occurs. You can use either Rhai scripts or coprocessors to add custom logic for what to log and when.

See the Apollo Solutions ["Hello World" coprocessor](https://github.com/apollosolutions/example-coprocessor-helloworld) for an example of a coprocessor that simply logs the router's payload.

<SolutionsNote />

### Alternative cloud services

If you are deploying the router to a cloud service, you likely already have access to the raw HTTP logs through other services like load balancers. You should be able to find specific client request logs for a particular operation using the operation hash or trace ID. Refer to the docs for your cloud providers for more information.

Popular cloud provider links are provided below.

#### Amazon Web Services

- [AWS CloudWatch Logs](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/WhatIsCloudWatchLogs.html)
- [AWS Elastic Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-access-logs.html)

#### Google Cloud Platform

- [Google Cloud Observability](https://cloud.google.com/logging/docs/log-analytics)
- [Google Cloud Load Balancing](https://cloud.google.com/load-balancing/docs/l7-internal/monitoring)

#### Microsoft Azure

- [Azure App Service Logging](https://learn.microsoft.com/en-us/azure/app-service/troubleshoot-diagnostic-logs)
- [Azure Load Balancer](https://learn.microsoft.com/en-us/azure/load-balancer/monitor-load-balancer)
Loading