From 4f949d41da8100ea50043ac62539e4e6145bc673 Mon Sep 17 00:00:00 2001 From: Lorenzo Mangani Date: Wed, 17 Jan 2024 12:00:30 +0100 Subject: [PATCH 1/8] Update _sidebar.md --- docs/_sidebar.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/_sidebar.md b/docs/_sidebar.md index e5d564a..2bbab46 100644 --- a/docs/_sidebar.md +++ b/docs/_sidebar.md @@ -15,10 +15,12 @@ * [Logs](logs/ingestion.md "QRYN – Supported Components – LogQL for ClickHouse and beyond") * [Metrics](metrics/ingestion.md "QRYN – Supported Components – LogQL for ClickHouse and beyond") * [Telemetry](telemetry/ingestion.md "QRYN – Supported Components – LogQL for ClickHouse and beyond") + * [Profiling](profiling/ingestion.md "QRYN – Supported Components – LogQL for ClickHouse and beyond") * [Querying](getting-started.md "QRYN – Getting Started – LogQL for ClickHouse and beyond") * [Logs](logs/query "QRYN – Getting Started – LogQL for ClickHouse and beyond") * [Metrics](metrics/query "QRYN by Metrico.in") * [Telemetry](telemetry/query "QRYN by Metrico.in") + * [Profiling](profiling/query "QRYN by Metrico.in") * [Advanced](examples.md "QRYN – Getting Started – LogQL for ClickHouse and beyond") * [LogQL](guide/logql.md "QRYN – Getting Started – LogQL for ClickHouse and beyond") * [APIs](support.md "QRYN – Supported Components – LogQL for ClickHouse and beyond") From 5ccd6ae1f17f0411c1d40009a6c19e5dce17bb2a Mon Sep 17 00:00:00 2001 From: Lorenzo Mangani Date: Wed, 17 Jan 2024 12:03:16 +0100 Subject: [PATCH 2/8] Pyroscope Ingestion placeholder --- docs/profiling/ingestion.md | 153 ++++++++++++++++++++++++++++++++++++ 1 file changed, 153 insertions(+) create mode 100644 docs/profiling/ingestion.md diff --git a/docs/profiling/ingestion.md b/docs/profiling/ingestion.md new file mode 100644 index 0000000..967c836 --- /dev/null +++ b/docs/profiling/ingestion.md @@ -0,0 +1,153 @@ +# 🔻 Profiling Ingestion + +The following protocol APIs are supported for ingesting continuous profiling events: + + +## ** OTEL Collector ** + + + +![image](https://user-images.githubusercontent.com/1423657/196469086-3d85efd5-7ef9-4d42-a677-5591470b7cae.png ':size=200') + +The [OTEL Collector]([https://opentelemetry.io/docs/collector/](https://github.com/metrico/otel-collector)) offers a vendor-agnostic implementation of how to receive, process and export telemetry data. It removes the need to run, operate, and maintain multiple agents/collectors. + +This works with improved scalability and supports a large variety of open-source observability data formats _(e.g. Jaeger, Prometheus, Fluent Bit, etc.)_ and allows aggregating and sending traces to **qryn** using the _Tempo API_ + +For large scale ingestion, we suggest using the dedicated [qryn opentelemetry distribution](https://github.com/metrico/otel-collector) for ClickHouse. + +### Examples + +![image](https://user-images.githubusercontent.com/1423657/217615085-cb5c7858-798e-4896-8ad6-3b88cfb16c6c.png) + + +#### qryn collector +``` +otel-collector: + container_name: otel-collector + image: ghcr.io/metrico/qryn-otel-collector:latest + volumes: + - ./otel-collector-config.yaml:/etc/otel/config.yaml + ports: + - "4317:4317" # OTLP gRPC receiver + - "4318:4318" # OTLP HTTP receiver + - "14250:14250" # Jaeger gRPC + - "14268:14268" # Jaeger thrift HTTP + - "9411:9411" # Zipkin port + - "24224:24224". # Fluent Forward + restart: on-failure +``` + +###### qryn collector `config.taml` +The following example enables all the supported formats at once. Filter the required sections to build your configuration. + +```yml +receivers: + otlp: + protocols: + grpc: + endpoint: 0.0.0.0:4317 + http: + endpoint: 0.0.0.0:4318 + jaeger: + protocols: + grpc: + endpoint: 0.0.0.0:14250 + thrift_http: + endpoint: 0.0.0.0:14268 + zipkin: + endpoint: 0.0.0.0:9411 + fluentforward: + endpoint: 0.0.0.0:24224 + prometheus: + config: + scrape_configs: + - job_name: 'otel-collector' + scrape_interval: 5s + static_configs: + - targets: ['exporter:8080'] +processors: + batch: + send_batch_size: 10000 + timeout: 5s + memory_limiter: + check_interval: 2s + limit_mib: 1800 + spike_limit_mib: 500 + resourcedetection/system: + detectors: ['system'] + system: + hostname_sources: ['os'] + resource: + attributes: + - key: service.name + value: "serviceName" + action: upsert + spanmetrics: + metrics_exporter: otlp/spanmetrics + latency_histogram_buckets: [100us, 1ms, 2ms, 6ms, 10ms, 100ms, 250ms] + dimensions_cache_size: 1500 + servicegraph: + metrics_exporter: otlp/spanmetrics + latency_histogram_buckets: [100us, 1ms, 2ms, 6ms, 10ms, 100ms, 250ms] + dimensions: [cluster, namespace] + store: + ttl: 2s + max_items: 200 + metricstransform: + transforms: + - include: calls_total + action: update + new_name: traces_spanmetrics_calls_total + - include: latency + action: update + new_name: traces_spanmetrics_latency +exporters: + qryn: + dsn: tcp://clickhouse-server:9000/cloki?username=default&password=************* + timeout: 10s + sending_queue: + queue_size: 100 + retry_on_failure: + enabled: true + initial_interval: 5s + max_interval: 30s + max_elapsed_time: 300s + logs: + format: json + otlp/spanmetrics: + endpoint: localhost:4317 + tls: + insecure: true +extensions: + health_check: + pprof: + zpages: + memory_ballast: + size_mib: 1000 + +service: + extensions: [pprof, zpages, health_check] + pipelines: + logs: + receivers: [fluentforward, otlp] + processors: [memory_limiter, resourcedetection/system, resource, batch] + exporters: [qryn] + traces: + receivers: [otlp, jaeger, zipkin] + processors: [memory_limiter, resourcedetection/system, resource, spanmetrics, servicegraph, batch] + exporters: [qryn] + # for align with https://grafana.com/docs/tempo/latest/metrics-generator/span_metrics/#how-to-run + metrics/spanmetrics: + receivers: [otlp] + processors: [metricstransform] + exporters: [qryn] + metrics: + receivers: [prometheus] + processors: [memory_limiter, resourcedetection/system, resource, batch] + exporters: [qryn] +``` + +?> _That's it!_ You're now _continuous profiling to **qryn** using OTLP Collector! + + + From 9849ff06c0ca7ef784beba95c5a7c00c6ad3c0e3 Mon Sep 17 00:00:00 2001 From: Lorenzo Mangani Date: Wed, 17 Jan 2024 12:07:33 +0100 Subject: [PATCH 3/8] Create query.md --- docs/profiling/query.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) create mode 100644 docs/profiling/query.md diff --git a/docs/profiling/query.md b/docs/profiling/query.md new file mode 100644 index 0000000..07712ee --- /dev/null +++ b/docs/profiling/query.md @@ -0,0 +1,21 @@ +# 🔎 Querying Profiling Data + +In this section, we'll learn how to query and filter profiling data using **Pyroscope**. Let's get __qryn__! + + +### ** ⭐ Grafana Pyroscope ** + + +Let's **explore** profiling data using [Grafana](guide/datasources.md) and its _flame visualizations_ + +### Pyroscope +#### Find Profiling data using Pyroscope 🧲 + +Pyroscope is _Grafana's Continuous Profiling_ stack providing ingestion and filtering capabilities. + +#### Search +Use the **qryn** `Pyroscope` datasource to find profiling data using _Search_ + +![screencast-localhost_3000-2024 01 16-12_43_50-ezgif com-video-to-gif-converter](https://github.com/metrico/qryn-docs/assets/1423657/3fe7167e-504a-42c0-bf51-bdb090ce0f6b) + + From bac002d1e86759c2ce2640bf2759d167c8476e4c Mon Sep 17 00:00:00 2001 From: Lorenzo Mangani Date: Wed, 17 Jan 2024 12:41:16 +0100 Subject: [PATCH 4/8] Update home.md --- docs/home.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/home.md b/docs/home.md index 8060e63..631f397 100644 --- a/docs/home.md +++ b/docs/home.md @@ -2,7 +2,10 @@

qryn: polyglot monitoring and observability

- + + + +

?> ... it's pronounced /ˈkwɪr..ɪŋ/ or just _querying_ From 621d7aa6780c0428e670df0fc8dc0d3294e9fd60 Mon Sep 17 00:00:00 2001 From: Lorenzo Mangani Date: Fri, 19 Jan 2024 14:36:43 +0100 Subject: [PATCH 5/8] Update ingestion.md --- docs/profiling/ingestion.md | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/docs/profiling/ingestion.md b/docs/profiling/ingestion.md index 967c836..9a4fa07 100644 --- a/docs/profiling/ingestion.md +++ b/docs/profiling/ingestion.md @@ -11,15 +11,12 @@ The following protocol APIs are supported for ingesting continuous profiling eve The [OTEL Collector]([https://opentelemetry.io/docs/collector/](https://github.com/metrico/otel-collector)) offers a vendor-agnostic implementation of how to receive, process and export telemetry data. It removes the need to run, operate, and maintain multiple agents/collectors. -This works with improved scalability and supports a large variety of open-source observability data formats _(e.g. Jaeger, Prometheus, Fluent Bit, etc.)_ and allows aggregating and sending traces to **qryn** using the _Tempo API_ +This works with improved scalability and supports a large variety of open-source observability data formats _(e.g. pprof, etc.)_ and allows aggregating and sending profiling data to **qryn** using the _Pyroscope API_ For large scale ingestion, we suggest using the dedicated [qryn opentelemetry distribution](https://github.com/metrico/otel-collector) for ClickHouse. ### Examples -![image](https://user-images.githubusercontent.com/1423657/217615085-cb5c7858-798e-4896-8ad6-3b88cfb16c6c.png) - - #### qryn collector ``` otel-collector: From 2e943610d0208daa45e19fd6efb503bc61eb4fe6 Mon Sep 17 00:00:00 2001 From: Lorenzo Mangani Date: Thu, 25 Jan 2024 11:28:59 +0100 Subject: [PATCH 6/8] Update query.md --- docs/profiling/query.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/docs/profiling/query.md b/docs/profiling/query.md index 07712ee..95ee761 100644 --- a/docs/profiling/query.md +++ b/docs/profiling/query.md @@ -6,7 +6,12 @@ In this section, we'll learn how to query and filter profiling data using **Pyro ### ** ⭐ Grafana Pyroscope ** -Let's **explore** profiling data using [Grafana](guide/datasources.md) and its _flame visualizations_ +Configure a new Pyroscope `datasource` using the Phlare format pointed at **qryn** + +![image](https://github.com/metrico/qryn-docs/assets/1423657/5c4f985c-c9fc-4676-8954-ac6973e77e9e) + +You are now ready to **explore** profiling data using Pyroscope and _flame visualizations_ + ### Pyroscope #### Find Profiling data using Pyroscope 🧲 From cb0e9d8bac73f0becb945c9bc5398a1285f92a5e Mon Sep 17 00:00:00 2001 From: Lorenzo Mangani Date: Thu, 25 Jan 2024 11:35:05 +0100 Subject: [PATCH 7/8] Update ingestion.md --- docs/profiling/ingestion.md | 119 +++--------------------------------- 1 file changed, 10 insertions(+), 109 deletions(-) diff --git a/docs/profiling/ingestion.md b/docs/profiling/ingestion.md index 9a4fa07..fb0f35a 100644 --- a/docs/profiling/ingestion.md +++ b/docs/profiling/ingestion.md @@ -13,7 +13,7 @@ The [OTEL Collector]([https://opentelemetry.io/docs/collector/](https://github.c This works with improved scalability and supports a large variety of open-source observability data formats _(e.g. pprof, etc.)_ and allows aggregating and sending profiling data to **qryn** using the _Pyroscope API_ -For large scale ingestion, we suggest using the dedicated [qryn opentelemetry distribution](https://github.com/metrico/otel-collector) for ClickHouse. +This guide is based on the [qryn opentelemetry distribution](https://github.com/metrico/otel-collector) for qryn + clickhouse. ### Examples @@ -25,126 +25,27 @@ otel-collector: volumes: - ./otel-collector-config.yaml:/etc/otel/config.yaml ports: - - "4317:4317" # OTLP gRPC receiver - - "4318:4318" # OTLP HTTP receiver - - "14250:14250" # Jaeger gRPC - - "14268:14268" # Jaeger thrift HTTP - - "9411:9411" # Zipkin port - - "24224:24224". # Fluent Forward + - "8062 :8062 " # Pyroscope gRPC receiver restart: on-failure ``` -###### qryn collector `config.taml` -The following example enables all the supported formats at once. Filter the required sections to build your configuration. +###### pyroscope `otel-collector-config.taml` +The following example enables Pyroscope ingestion using the qryn-collector. Integrate in your existing configuration. ```yml receivers: - otlp: - protocols: - grpc: - endpoint: 0.0.0.0:4317 - http: - endpoint: 0.0.0.0:4318 - jaeger: - protocols: - grpc: - endpoint: 0.0.0.0:14250 - thrift_http: - endpoint: 0.0.0.0:14268 - zipkin: - endpoint: 0.0.0.0:9411 - fluentforward: - endpoint: 0.0.0.0:24224 - prometheus: - config: - scrape_configs: - - job_name: 'otel-collector' - scrape_interval: 5s - static_configs: - - targets: ['exporter:8080'] -processors: - batch: - send_batch_size: 10000 - timeout: 5s - memory_limiter: - check_interval: 2s - limit_mib: 1800 - spike_limit_mib: 500 - resourcedetection/system: - detectors: ['system'] - system: - hostname_sources: ['os'] - resource: - attributes: - - key: service.name - value: "serviceName" - action: upsert - spanmetrics: - metrics_exporter: otlp/spanmetrics - latency_histogram_buckets: [100us, 1ms, 2ms, 6ms, 10ms, 100ms, 250ms] - dimensions_cache_size: 1500 - servicegraph: - metrics_exporter: otlp/spanmetrics - latency_histogram_buckets: [100us, 1ms, 2ms, 6ms, 10ms, 100ms, 250ms] - dimensions: [cluster, namespace] - store: - ttl: 2s - max_items: 200 - metricstransform: - transforms: - - include: calls_total - action: update - new_name: traces_spanmetrics_calls_total - - include: latency - action: update - new_name: traces_spanmetrics_latency + pyroscopereceiver: exporters: - qryn: - dsn: tcp://clickhouse-server:9000/cloki?username=default&password=************* - timeout: 10s - sending_queue: - queue_size: 100 - retry_on_failure: - enabled: true - initial_interval: 5s - max_interval: 30s - max_elapsed_time: 300s - logs: - format: json - otlp/spanmetrics: - endpoint: localhost:4317 - tls: - insecure: true -extensions: - health_check: - pprof: - zpages: - memory_ballast: - size_mib: 1000 - + clickhouseprofileexporter: + dsn: clickhouse://localhost:9000/qryn service: - extensions: [pprof, zpages, health_check] pipelines: logs: - receivers: [fluentforward, otlp] - processors: [memory_limiter, resourcedetection/system, resource, batch] - exporters: [qryn] - traces: - receivers: [otlp, jaeger, zipkin] - processors: [memory_limiter, resourcedetection/system, resource, spanmetrics, servicegraph, batch] - exporters: [qryn] - # for align with https://grafana.com/docs/tempo/latest/metrics-generator/span_metrics/#how-to-run - metrics/spanmetrics: - receivers: [otlp] - processors: [metricstransform] - exporters: [qryn] - metrics: - receivers: [prometheus] - processors: [memory_limiter, resourcedetection/system, resource, batch] - exporters: [qryn] + receivers: [pyroscopereceiver] + exporters: [clickhouseprofileexporter] ``` -?> _That's it!_ You're now _continuous profiling to **qryn** using OTLP Collector! +?> _That's it!_ You're now ready to ingest _continuous profiling into **qryn** using OTLP Collector! From 7611ca33d92e290fde3753c7a7b2d45a18123cc8 Mon Sep 17 00:00:00 2001 From: Lorenzo Mangani Date: Thu, 25 Jan 2024 13:40:17 +0100 Subject: [PATCH 8/8] Java Link --- docs/profiling/ingestion.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/docs/profiling/ingestion.md b/docs/profiling/ingestion.md index fb0f35a..5e0a072 100644 --- a/docs/profiling/ingestion.md +++ b/docs/profiling/ingestion.md @@ -47,5 +47,17 @@ service: ?> _That's it!_ You're now ready to ingest _continuous profiling into **qryn** using OTLP Collector! +## ** Pyroscope Java ** + + + +You can use qryn with the [Pyroscope Java](https://github.com/grafana/pyroscope-java) client + +The agent is distributed as a single JAR file `pyroscope.jar` containing native async-profiler libraries for: + +* Linux on x64 +* Linux on ARM64 +* MacOS on x64 +* MacOS on ARM64