Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[APM] Fix missing error.culprit #205242

Merged
merged 1 commit into from
Dec 30, 2024

Conversation

crespocarlos
Copy link
Contributor

@crespocarlos crespocarlos commented Dec 30, 2024

fixes #205240

Summary

Fixes the issue where error.culprit is always missing.

image image

How to test

  • Run simple_trace synthtrace scenario
  • Run the request below
POST logs-apm.error-default/_doc
POST logs-apm.error-default/_doc
{

    "container": {
      "id": "80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359"
    },
    "kubernetes": {
      "node": {
        "name": "ip-192-168-55-136.ec2.internal"
      },
      "pod": {
        "uid": "9bbfe3dc-dace-4638-b8f5-aa6ccf435cbe",
        "name": "otel-demo-green-3-cartservice-5d5ccc86dd-gcxtr"
      },
      "namespace": "otel-demo-green-3"
    },
    "parent": {
      "id": "f39639f56a540e0e"
    },
    "agent": {
      "name": "opentelemetry/dotnet",
      "version": "1.6.0"
    },
    "source": {
      "port": 35136,
      "ip": "::ffff:192.168.50.132"
    },
    "error": {
      "exception": [
        {
          "handled": true,
          "message": "Exception (plain, .NET) without culprit",
          "type": "Grpc.Core.RpcException"
        }
      ],
      "culprit": "culprit msg", // remove to test the N/A scenario
      "id": "60674e9b23c8b06d83889e63ffe82c1c", //change the id on every run
      "stack_trace": "Grpc.Core.RpcException: Status(StatusCode=\"FailedPrecondition\", Detail=\"Can't access cart storage. System.ApplicationException: Wasn't able to connect to redis\n   at cartservice.cartstore.RedisCartStore.EnsureRedisConnected() in /usr/src/app/src/cartstore/RedisCartStore.cs:line 79\n   at cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in /usr/src/app/src/cartstore/RedisCartStore.cs:line 157\")\n   at cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in /usr/src/app/src/cartstore/RedisCartStore.cs:line 166\n   at cartservice.services.CartService.EmptyCart(EmptyCartRequest request, ServerCallContext context) in /usr/src/app/src/services/CartService.cs:line 65",
      "grouping_key": "6082c58afd7fa6838ebc27eef6e04829"
    },
    "message": "Exception (plain, .NET)",
    "url": {
      "path": "/oteldemo.CartService/EmptyCart",
      "original": "http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart",
      "scheme": "http",
      "port": 8080,
      "domain": "otel-demo-green-3-cartservice",
      "full": "http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart"
    },
    "labels": {
      "k8s_pod_ip": "192.168.62.174",
      "k8s_pod_start_time": "2024-12-17T02:55:54Z",
      "service_namespace": "opentelemetry-demo",
      "k8s_deployment_name": "otel-demo-green-3-cartservice"
    },
    "observer": {
      "hostname": "450fbf7ab701",
      "type": "apm-server",
      "version": "8.13.2"
    },
    "trace": {
      "id": "699f6d1039fc873191a1c983790db3a2"
    },
    "@timestamp": "2024-12-30T10:30:00.000Z",
    "data_stream": {
      "namespace": "default",
      "type": "logs",
      "dataset": "apm.error"
    },
    "service": {
      "node": {
        "name": "80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359"
      },
      "framework": {
        "name": "Microsoft.AspNetCore"
      },
       "name": "synth-node-0",
      "language": {
        "name": "dotnet"
      }
    },
    "host": {
      "hostname": "ip-192-168-55-136.ec2.internal",
      "name": "ip-192-168-55-136.ec2.internal"
    },
    "client": {
      "port": 35136,
      "ip": "::ffff:192.168.50.132"
    },
    "http": {
      "request": {
        "method": "POST"
      },
      "response": {
        "status_code": 200
      },
      "version": "2"
    },
    "event": {
      "agent_id_status": "missing",
      "ingested": "2024-12-18T18:04:22Z"
    },
    "transaction": {
      "id": "f39639f56a540e0e",
      "type": "request",
      "sampled": true
    },
    "user_agent": {
      "original": "grpc-go/1.59.0",
      "name": "Other",
      "device": {
        "name": "Other"
      }
    },
    "span": {
      "id": "f39639f56a540e0e"
    },
    "timestamp": {
      "us": 1704892605838000
    }
  }

@crespocarlos
Copy link
Contributor Author

/ci

@crespocarlos crespocarlos added backport:skip This commit does not require backporting v8.16.0 backport:version Backport to applied version labels v8.17.0 release_note:skip Skip the PR/issue when compiling release notes Team:obs-ux-infra_services Observability Infrastructure & Services User Experience Team backport:prev-major Backport to (8.x, 8.17, 8.16) the previous major branch and other branches in development and removed backport:skip This commit does not require backporting v8.16.0 v8.17.0 backport:version Backport to applied version labels labels Dec 30, 2024
@crespocarlos crespocarlos marked this pull request as ready for review December 30, 2024 10:44
@crespocarlos crespocarlos requested a review from a team as a code owner December 30, 2024 10:44
@elasticmachine
Copy link
Contributor

Pinging @elastic/obs-ux-infra_services-team (Team:obs-ux-infra_services)

Copy link
Contributor

@Bluefinger Bluefinger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@crespocarlos crespocarlos merged commit 12608c3 into elastic:main Dec 30, 2024
27 checks passed
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.16, 8.17, 8.x

https://github.com/elastic/kibana/actions/runs/12546786493

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Dec 30, 2024
fixes [elastic#205240](elastic#205240)

## Summary

Fixes the issue where `error.culprit` is always missing.

<img width="800" alt="image"
src="https://github.com/user-attachments/assets/9dad6797-dd40-4206-b508-daf8d69d653d"
/>

<img width="800" alt="image"
src="https://github.com/user-attachments/assets/6a0ac25a-bc4b-4a26-b126-86eecbe7e769"
/>

### How to test

- Run `simple_trace` synthtrace scenario
- Run the request below
<details>
<summary>POST logs-apm.error-default/_doc</summary>

```
POST logs-apm.error-default/_doc
{

    "container": {
      "id": "80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359"
    },
    "kubernetes": {
      "node": {
        "name": "ip-192-168-55-136.ec2.internal"
      },
      "pod": {
        "uid": "9bbfe3dc-dace-4638-b8f5-aa6ccf435cbe",
        "name": "otel-demo-green-3-cartservice-5d5ccc86dd-gcxtr"
      },
      "namespace": "otel-demo-green-3"
    },
    "parent": {
      "id": "f39639f56a540e0e"
    },
    "agent": {
      "name": "opentelemetry/dotnet",
      "version": "1.6.0"
    },
    "source": {
      "port": 35136,
      "ip": "::ffff:192.168.50.132"
    },
    "error": {
      "exception": [
        {
          "handled": true,
          "message": "Exception (plain, .NET) without culprit",
          "type": "Grpc.Core.RpcException"
        }
      ],
      "culprit": "culprit msg", // remove to test the N/A scenario
      "id": "60674e9b23c8b06d83889e63ffe82c1c", //change the id on every run
      "stack_trace": "Grpc.Core.RpcException: Status(StatusCode=\"FailedPrecondition\", Detail=\"Can't access cart storage. System.ApplicationException: Wasn't able to connect to redis\n   at cartservice.cartstore.RedisCartStore.EnsureRedisConnected() in /usr/src/app/src/cartstore/RedisCartStore.cs:line 79\n   at cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in /usr/src/app/src/cartstore/RedisCartStore.cs:line 157\")\n   at cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in /usr/src/app/src/cartstore/RedisCartStore.cs:line 166\n   at cartservice.services.CartService.EmptyCart(EmptyCartRequest request, ServerCallContext context) in /usr/src/app/src/services/CartService.cs:line 65",
      "grouping_key": "6082c58afd7fa6838ebc27eef6e04829"
    },
    "message": "Exception (plain, .NET)",
    "url": {
      "path": "/oteldemo.CartService/EmptyCart",
      "original": "http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart",
      "scheme": "http",
      "port": 8080,
      "domain": "otel-demo-green-3-cartservice",
      "full": "http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart"
    },
    "labels": {
      "k8s_pod_ip": "192.168.62.174",
      "k8s_pod_start_time": "2024-12-17T02:55:54Z",
      "service_namespace": "opentelemetry-demo",
      "k8s_deployment_name": "otel-demo-green-3-cartservice"
    },
    "observer": {
      "hostname": "450fbf7ab701",
      "type": "apm-server",
      "version": "8.13.2"
    },
    "trace": {
      "id": "699f6d1039fc873191a1c983790db3a2"
    },
    "@timestamp": "2024-12-30T10:30:00.000Z",
    "data_stream": {
      "namespace": "default",
      "type": "logs",
      "dataset": "apm.error"
    },
    "service": {
      "node": {
        "name": "80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359"
      },
      "framework": {
        "name": "Microsoft.AspNetCore"
      },
       "name": "synth-node-0",
      "language": {
        "name": "dotnet"
      }
    },
    "host": {
      "hostname": "ip-192-168-55-136.ec2.internal",
      "name": "ip-192-168-55-136.ec2.internal"
    },
    "client": {
      "port": 35136,
      "ip": "::ffff:192.168.50.132"
    },
    "http": {
      "request": {
        "method": "POST"
      },
      "response": {
        "status_code": 200
      },
      "version": "2"
    },
    "event": {
      "agent_id_status": "missing",
      "ingested": "2024-12-18T18:04:22Z"
    },
    "transaction": {
      "id": "f39639f56a540e0e",
      "type": "request",
      "sampled": true
    },
    "user_agent": {
      "original": "grpc-go/1.59.0",
      "name": "Other",
      "device": {
        "name": "Other"
      }
    },
    "span": {
      "id": "f39639f56a540e0e"
    },
    "timestamp": {
      "us": 1704892605838000
    }
  }
```
</details>

(cherry picked from commit 12608c3)
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Dec 30, 2024
fixes [elastic#205240](elastic#205240)

## Summary

Fixes the issue where `error.culprit` is always missing.

<img width="800" alt="image"
src="https://github.com/user-attachments/assets/9dad6797-dd40-4206-b508-daf8d69d653d"
/>

<img width="800" alt="image"
src="https://github.com/user-attachments/assets/6a0ac25a-bc4b-4a26-b126-86eecbe7e769"
/>

### How to test

- Run `simple_trace` synthtrace scenario
- Run the request below
<details>
<summary>POST logs-apm.error-default/_doc</summary>

```
POST logs-apm.error-default/_doc
{

    "container": {
      "id": "80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359"
    },
    "kubernetes": {
      "node": {
        "name": "ip-192-168-55-136.ec2.internal"
      },
      "pod": {
        "uid": "9bbfe3dc-dace-4638-b8f5-aa6ccf435cbe",
        "name": "otel-demo-green-3-cartservice-5d5ccc86dd-gcxtr"
      },
      "namespace": "otel-demo-green-3"
    },
    "parent": {
      "id": "f39639f56a540e0e"
    },
    "agent": {
      "name": "opentelemetry/dotnet",
      "version": "1.6.0"
    },
    "source": {
      "port": 35136,
      "ip": "::ffff:192.168.50.132"
    },
    "error": {
      "exception": [
        {
          "handled": true,
          "message": "Exception (plain, .NET) without culprit",
          "type": "Grpc.Core.RpcException"
        }
      ],
      "culprit": "culprit msg", // remove to test the N/A scenario
      "id": "60674e9b23c8b06d83889e63ffe82c1c", //change the id on every run
      "stack_trace": "Grpc.Core.RpcException: Status(StatusCode=\"FailedPrecondition\", Detail=\"Can't access cart storage. System.ApplicationException: Wasn't able to connect to redis\n   at cartservice.cartstore.RedisCartStore.EnsureRedisConnected() in /usr/src/app/src/cartstore/RedisCartStore.cs:line 79\n   at cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in /usr/src/app/src/cartstore/RedisCartStore.cs:line 157\")\n   at cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in /usr/src/app/src/cartstore/RedisCartStore.cs:line 166\n   at cartservice.services.CartService.EmptyCart(EmptyCartRequest request, ServerCallContext context) in /usr/src/app/src/services/CartService.cs:line 65",
      "grouping_key": "6082c58afd7fa6838ebc27eef6e04829"
    },
    "message": "Exception (plain, .NET)",
    "url": {
      "path": "/oteldemo.CartService/EmptyCart",
      "original": "http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart",
      "scheme": "http",
      "port": 8080,
      "domain": "otel-demo-green-3-cartservice",
      "full": "http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart"
    },
    "labels": {
      "k8s_pod_ip": "192.168.62.174",
      "k8s_pod_start_time": "2024-12-17T02:55:54Z",
      "service_namespace": "opentelemetry-demo",
      "k8s_deployment_name": "otel-demo-green-3-cartservice"
    },
    "observer": {
      "hostname": "450fbf7ab701",
      "type": "apm-server",
      "version": "8.13.2"
    },
    "trace": {
      "id": "699f6d1039fc873191a1c983790db3a2"
    },
    "@timestamp": "2024-12-30T10:30:00.000Z",
    "data_stream": {
      "namespace": "default",
      "type": "logs",
      "dataset": "apm.error"
    },
    "service": {
      "node": {
        "name": "80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359"
      },
      "framework": {
        "name": "Microsoft.AspNetCore"
      },
       "name": "synth-node-0",
      "language": {
        "name": "dotnet"
      }
    },
    "host": {
      "hostname": "ip-192-168-55-136.ec2.internal",
      "name": "ip-192-168-55-136.ec2.internal"
    },
    "client": {
      "port": 35136,
      "ip": "::ffff:192.168.50.132"
    },
    "http": {
      "request": {
        "method": "POST"
      },
      "response": {
        "status_code": 200
      },
      "version": "2"
    },
    "event": {
      "agent_id_status": "missing",
      "ingested": "2024-12-18T18:04:22Z"
    },
    "transaction": {
      "id": "f39639f56a540e0e",
      "type": "request",
      "sampled": true
    },
    "user_agent": {
      "original": "grpc-go/1.59.0",
      "name": "Other",
      "device": {
        "name": "Other"
      }
    },
    "span": {
      "id": "f39639f56a540e0e"
    },
    "timestamp": {
      "us": 1704892605838000
    }
  }
```
</details>

(cherry picked from commit 12608c3)
@kibanamachine
Copy link
Contributor

💔 Some backports could not be created

Status Branch Result
8.16 Backport failed because of merge conflicts

You might need to backport the following PRs to 8.16:
- [ci] Update VM cache for each ES snapshot (#205119)
- Fixes for storybook aliases (#204842)
8.17
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Manual backport

To create the backport manually run:

node scripts/backport --pr 205242

Questions ?

Please refer to the Backport tool documentation

crespocarlos added a commit to crespocarlos/kibana that referenced this pull request Dec 30, 2024
fixes [elastic#205240](elastic#205240)

## Summary

Fixes the issue where `error.culprit` is always missing.

<img width="800" alt="image"
src="https://github.com/user-attachments/assets/9dad6797-dd40-4206-b508-daf8d69d653d"
/>

<img width="800" alt="image"
src="https://github.com/user-attachments/assets/6a0ac25a-bc4b-4a26-b126-86eecbe7e769"
/>

### How to test

- Run `simple_trace` synthtrace scenario
- Run the request below
<details>
<summary>POST logs-apm.error-default/_doc</summary>

```
POST logs-apm.error-default/_doc
{

    "container": {
      "id": "80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359"
    },
    "kubernetes": {
      "node": {
        "name": "ip-192-168-55-136.ec2.internal"
      },
      "pod": {
        "uid": "9bbfe3dc-dace-4638-b8f5-aa6ccf435cbe",
        "name": "otel-demo-green-3-cartservice-5d5ccc86dd-gcxtr"
      },
      "namespace": "otel-demo-green-3"
    },
    "parent": {
      "id": "f39639f56a540e0e"
    },
    "agent": {
      "name": "opentelemetry/dotnet",
      "version": "1.6.0"
    },
    "source": {
      "port": 35136,
      "ip": "::ffff:192.168.50.132"
    },
    "error": {
      "exception": [
        {
          "handled": true,
          "message": "Exception (plain, .NET) without culprit",
          "type": "Grpc.Core.RpcException"
        }
      ],
      "culprit": "culprit msg", // remove to test the N/A scenario
      "id": "60674e9b23c8b06d83889e63ffe82c1c", //change the id on every run
      "stack_trace": "Grpc.Core.RpcException: Status(StatusCode=\"FailedPrecondition\", Detail=\"Can't access cart storage. System.ApplicationException: Wasn't able to connect to redis\n   at cartservice.cartstore.RedisCartStore.EnsureRedisConnected() in /usr/src/app/src/cartstore/RedisCartStore.cs:line 79\n   at cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in /usr/src/app/src/cartstore/RedisCartStore.cs:line 157\")\n   at cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in /usr/src/app/src/cartstore/RedisCartStore.cs:line 166\n   at cartservice.services.CartService.EmptyCart(EmptyCartRequest request, ServerCallContext context) in /usr/src/app/src/services/CartService.cs:line 65",
      "grouping_key": "6082c58afd7fa6838ebc27eef6e04829"
    },
    "message": "Exception (plain, .NET)",
    "url": {
      "path": "/oteldemo.CartService/EmptyCart",
      "original": "http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart",
      "scheme": "http",
      "port": 8080,
      "domain": "otel-demo-green-3-cartservice",
      "full": "http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart"
    },
    "labels": {
      "k8s_pod_ip": "192.168.62.174",
      "k8s_pod_start_time": "2024-12-17T02:55:54Z",
      "service_namespace": "opentelemetry-demo",
      "k8s_deployment_name": "otel-demo-green-3-cartservice"
    },
    "observer": {
      "hostname": "450fbf7ab701",
      "type": "apm-server",
      "version": "8.13.2"
    },
    "trace": {
      "id": "699f6d1039fc873191a1c983790db3a2"
    },
    "@timestamp": "2024-12-30T10:30:00.000Z",
    "data_stream": {
      "namespace": "default",
      "type": "logs",
      "dataset": "apm.error"
    },
    "service": {
      "node": {
        "name": "80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359"
      },
      "framework": {
        "name": "Microsoft.AspNetCore"
      },
       "name": "synth-node-0",
      "language": {
        "name": "dotnet"
      }
    },
    "host": {
      "hostname": "ip-192-168-55-136.ec2.internal",
      "name": "ip-192-168-55-136.ec2.internal"
    },
    "client": {
      "port": 35136,
      "ip": "::ffff:192.168.50.132"
    },
    "http": {
      "request": {
        "method": "POST"
      },
      "response": {
        "status_code": 200
      },
      "version": "2"
    },
    "event": {
      "agent_id_status": "missing",
      "ingested": "2024-12-18T18:04:22Z"
    },
    "transaction": {
      "id": "f39639f56a540e0e",
      "type": "request",
      "sampled": true
    },
    "user_agent": {
      "original": "grpc-go/1.59.0",
      "name": "Other",
      "device": {
        "name": "Other"
      }
    },
    "span": {
      "id": "f39639f56a540e0e"
    },
    "timestamp": {
      "us": 1704892605838000
    }
  }
```
</details>

(cherry picked from commit 12608c3)

# Conflicts:
#	x-pack/test/api_integration/deployment_agnostic/apis/observability/apm/mobile/errors/group_id_samples.spec.ts
@crespocarlos
Copy link
Contributor Author

💚 All backports created successfully

Status Branch Result
8.16

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

@crespocarlos crespocarlos changed the title [APM Fix missing error.culprit [APM] Fix missing error.culprit Dec 30, 2024
kibanamachine added a commit that referenced this pull request Dec 30, 2024
# Backport

This will backport the following commits from `main` to `8.17`:
- [[APM Fix missing error.culprit
(#205242)](#205242)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Carlos
Crespo","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-12-30T12:50:48Z","message":"[APM
Fix missing error.culprit (#205242)\n\nfixes
[#205240](https://github.com/elastic/kibana/issues/205240)\r\n\r\n##
Summary\r\n\r\nFixes the issue where `error.culprit` is always
missing.\r\n\r\n<img width=\"800\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/9dad6797-dd40-4206-b508-daf8d69d653d\"\r\n/>\r\n\r\n<img
width=\"800\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/6a0ac25a-bc4b-4a26-b126-86eecbe7e769\"\r\n/>\r\n\r\n\r\n\r\n###
How to test \r\n\r\n- Run `simple_trace` synthtrace scenario\r\n- Run
the request below\r\n<details>\r\n<summary>POST
logs-apm.error-default/_doc</summary>\r\n\r\n```\r\nPOST
logs-apm.error-default/_doc\r\n{\r\n\r\n \"container\": {\r\n \"id\":
\"80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359\"\r\n
},\r\n \"kubernetes\": {\r\n \"node\": {\r\n \"name\":
\"ip-192-168-55-136.ec2.internal\"\r\n },\r\n \"pod\": {\r\n \"uid\":
\"9bbfe3dc-dace-4638-b8f5-aa6ccf435cbe\",\r\n \"name\":
\"otel-demo-green-3-cartservice-5d5ccc86dd-gcxtr\"\r\n },\r\n
\"namespace\": \"otel-demo-green-3\"\r\n },\r\n \"parent\": {\r\n
\"id\": \"f39639f56a540e0e\"\r\n },\r\n \"agent\": {\r\n \"name\":
\"opentelemetry/dotnet\",\r\n \"version\": \"1.6.0\"\r\n },\r\n
\"source\": {\r\n \"port\": 35136,\r\n \"ip\":
\"::ffff:192.168.50.132\"\r\n },\r\n \"error\": {\r\n \"exception\":
[\r\n {\r\n \"handled\": true,\r\n \"message\": \"Exception (plain,
.NET) without culprit\",\r\n \"type\": \"Grpc.Core.RpcException\"\r\n
}\r\n ],\r\n \"culprit\": \"culprit msg\", // remove to test the N/A
scenario\r\n \"id\": \"60674e9b23c8b06d83889e63ffe82c1c\", //change the
id on every run\r\n \"stack_trace\": \"Grpc.Core.RpcException:
Status(StatusCode=\\\"FailedPrecondition\\\", Detail=\\\"Can't access
cart storage. System.ApplicationException: Wasn't able to connect to
redis\\n at cartservice.cartstore.RedisCartStore.EnsureRedisConnected()
in /usr/src/app/src/cartstore/RedisCartStore.cs:line 79\\n at
cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in
/usr/src/app/src/cartstore/RedisCartStore.cs:line 157\\\")\\n at
cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in
/usr/src/app/src/cartstore/RedisCartStore.cs:line 166\\n at
cartservice.services.CartService.EmptyCart(EmptyCartRequest request,
ServerCallContext context) in
/usr/src/app/src/services/CartService.cs:line 65\",\r\n
\"grouping_key\": \"6082c58afd7fa6838ebc27eef6e04829\"\r\n },\r\n
\"message\": \"Exception (plain, .NET)\",\r\n \"url\": {\r\n \"path\":
\"/oteldemo.CartService/EmptyCart\",\r\n \"original\":
\"http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart\",\r\n
\"scheme\": \"http\",\r\n \"port\": 8080,\r\n \"domain\":
\"otel-demo-green-3-cartservice\",\r\n \"full\":
\"http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart\"\r\n
},\r\n \"labels\": {\r\n \"k8s_pod_ip\": \"192.168.62.174\",\r\n
\"k8s_pod_start_time\": \"2024-12-17T02:55:54Z\",\r\n
\"service_namespace\": \"opentelemetry-demo\",\r\n
\"k8s_deployment_name\": \"otel-demo-green-3-cartservice\"\r\n },\r\n
\"observer\": {\r\n \"hostname\": \"450fbf7ab701\",\r\n \"type\":
\"apm-server\",\r\n \"version\": \"8.13.2\"\r\n },\r\n \"trace\": {\r\n
\"id\": \"699f6d1039fc873191a1c983790db3a2\"\r\n },\r\n \"@timestamp\":
\"2024-12-30T10:30:00.000Z\",\r\n \"data_stream\": {\r\n \"namespace\":
\"default\",\r\n \"type\": \"logs\",\r\n \"dataset\": \"apm.error\"\r\n
},\r\n \"service\": {\r\n \"node\": {\r\n \"name\":
\"80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359\"\r\n
},\r\n \"framework\": {\r\n \"name\": \"Microsoft.AspNetCore\"\r\n
},\r\n \"name\": \"synth-node-0\",\r\n \"language\": {\r\n \"name\":
\"dotnet\"\r\n }\r\n },\r\n \"host\": {\r\n \"hostname\":
\"ip-192-168-55-136.ec2.internal\",\r\n \"name\":
\"ip-192-168-55-136.ec2.internal\"\r\n },\r\n \"client\": {\r\n
\"port\": 35136,\r\n \"ip\": \"::ffff:192.168.50.132\"\r\n },\r\n
\"http\": {\r\n \"request\": {\r\n \"method\": \"POST\"\r\n },\r\n
\"response\": {\r\n \"status_code\": 200\r\n },\r\n \"version\":
\"2\"\r\n },\r\n \"event\": {\r\n \"agent_id_status\": \"missing\",\r\n
\"ingested\": \"2024-12-18T18:04:22Z\"\r\n },\r\n \"transaction\": {\r\n
\"id\": \"f39639f56a540e0e\",\r\n \"type\": \"request\",\r\n
\"sampled\": true\r\n },\r\n \"user_agent\": {\r\n \"original\":
\"grpc-go/1.59.0\",\r\n \"name\": \"Other\",\r\n \"device\": {\r\n
\"name\": \"Other\"\r\n }\r\n },\r\n \"span\": {\r\n \"id\":
\"f39639f56a540e0e\"\r\n },\r\n \"timestamp\": {\r\n \"us\":
1704892605838000\r\n }\r\n
}\r\n```\r\n</details>","sha":"12608c3659a2aaea702c1d341bd0fab30f968056","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","backport:prev-major","Team:obs-ux-infra_services"],"title":"[APM
Fix missing
error.culprit","number":205242,"url":"https://github.com/elastic/kibana/pull/205242","mergeCommit":{"message":"[APM
Fix missing error.culprit (#205242)\n\nfixes
[#205240](https://github.com/elastic/kibana/issues/205240)\r\n\r\n##
Summary\r\n\r\nFixes the issue where `error.culprit` is always
missing.\r\n\r\n<img width=\"800\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/9dad6797-dd40-4206-b508-daf8d69d653d\"\r\n/>\r\n\r\n<img
width=\"800\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/6a0ac25a-bc4b-4a26-b126-86eecbe7e769\"\r\n/>\r\n\r\n\r\n\r\n###
How to test \r\n\r\n- Run `simple_trace` synthtrace scenario\r\n- Run
the request below\r\n<details>\r\n<summary>POST
logs-apm.error-default/_doc</summary>\r\n\r\n```\r\nPOST
logs-apm.error-default/_doc\r\n{\r\n\r\n \"container\": {\r\n \"id\":
\"80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359\"\r\n
},\r\n \"kubernetes\": {\r\n \"node\": {\r\n \"name\":
\"ip-192-168-55-136.ec2.internal\"\r\n },\r\n \"pod\": {\r\n \"uid\":
\"9bbfe3dc-dace-4638-b8f5-aa6ccf435cbe\",\r\n \"name\":
\"otel-demo-green-3-cartservice-5d5ccc86dd-gcxtr\"\r\n },\r\n
\"namespace\": \"otel-demo-green-3\"\r\n },\r\n \"parent\": {\r\n
\"id\": \"f39639f56a540e0e\"\r\n },\r\n \"agent\": {\r\n \"name\":
\"opentelemetry/dotnet\",\r\n \"version\": \"1.6.0\"\r\n },\r\n
\"source\": {\r\n \"port\": 35136,\r\n \"ip\":
\"::ffff:192.168.50.132\"\r\n },\r\n \"error\": {\r\n \"exception\":
[\r\n {\r\n \"handled\": true,\r\n \"message\": \"Exception (plain,
.NET) without culprit\",\r\n \"type\": \"Grpc.Core.RpcException\"\r\n
}\r\n ],\r\n \"culprit\": \"culprit msg\", // remove to test the N/A
scenario\r\n \"id\": \"60674e9b23c8b06d83889e63ffe82c1c\", //change the
id on every run\r\n \"stack_trace\": \"Grpc.Core.RpcException:
Status(StatusCode=\\\"FailedPrecondition\\\", Detail=\\\"Can't access
cart storage. System.ApplicationException: Wasn't able to connect to
redis\\n at cartservice.cartstore.RedisCartStore.EnsureRedisConnected()
in /usr/src/app/src/cartstore/RedisCartStore.cs:line 79\\n at
cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in
/usr/src/app/src/cartstore/RedisCartStore.cs:line 157\\\")\\n at
cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in
/usr/src/app/src/cartstore/RedisCartStore.cs:line 166\\n at
cartservice.services.CartService.EmptyCart(EmptyCartRequest request,
ServerCallContext context) in
/usr/src/app/src/services/CartService.cs:line 65\",\r\n
\"grouping_key\": \"6082c58afd7fa6838ebc27eef6e04829\"\r\n },\r\n
\"message\": \"Exception (plain, .NET)\",\r\n \"url\": {\r\n \"path\":
\"/oteldemo.CartService/EmptyCart\",\r\n \"original\":
\"http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart\",\r\n
\"scheme\": \"http\",\r\n \"port\": 8080,\r\n \"domain\":
\"otel-demo-green-3-cartservice\",\r\n \"full\":
\"http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart\"\r\n
},\r\n \"labels\": {\r\n \"k8s_pod_ip\": \"192.168.62.174\",\r\n
\"k8s_pod_start_time\": \"2024-12-17T02:55:54Z\",\r\n
\"service_namespace\": \"opentelemetry-demo\",\r\n
\"k8s_deployment_name\": \"otel-demo-green-3-cartservice\"\r\n },\r\n
\"observer\": {\r\n \"hostname\": \"450fbf7ab701\",\r\n \"type\":
\"apm-server\",\r\n \"version\": \"8.13.2\"\r\n },\r\n \"trace\": {\r\n
\"id\": \"699f6d1039fc873191a1c983790db3a2\"\r\n },\r\n \"@timestamp\":
\"2024-12-30T10:30:00.000Z\",\r\n \"data_stream\": {\r\n \"namespace\":
\"default\",\r\n \"type\": \"logs\",\r\n \"dataset\": \"apm.error\"\r\n
},\r\n \"service\": {\r\n \"node\": {\r\n \"name\":
\"80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359\"\r\n
},\r\n \"framework\": {\r\n \"name\": \"Microsoft.AspNetCore\"\r\n
},\r\n \"name\": \"synth-node-0\",\r\n \"language\": {\r\n \"name\":
\"dotnet\"\r\n }\r\n },\r\n \"host\": {\r\n \"hostname\":
\"ip-192-168-55-136.ec2.internal\",\r\n \"name\":
\"ip-192-168-55-136.ec2.internal\"\r\n },\r\n \"client\": {\r\n
\"port\": 35136,\r\n \"ip\": \"::ffff:192.168.50.132\"\r\n },\r\n
\"http\": {\r\n \"request\": {\r\n \"method\": \"POST\"\r\n },\r\n
\"response\": {\r\n \"status_code\": 200\r\n },\r\n \"version\":
\"2\"\r\n },\r\n \"event\": {\r\n \"agent_id_status\": \"missing\",\r\n
\"ingested\": \"2024-12-18T18:04:22Z\"\r\n },\r\n \"transaction\": {\r\n
\"id\": \"f39639f56a540e0e\",\r\n \"type\": \"request\",\r\n
\"sampled\": true\r\n },\r\n \"user_agent\": {\r\n \"original\":
\"grpc-go/1.59.0\",\r\n \"name\": \"Other\",\r\n \"device\": {\r\n
\"name\": \"Other\"\r\n }\r\n },\r\n \"span\": {\r\n \"id\":
\"f39639f56a540e0e\"\r\n },\r\n \"timestamp\": {\r\n \"us\":
1704892605838000\r\n }\r\n
}\r\n```\r\n</details>","sha":"12608c3659a2aaea702c1d341bd0fab30f968056"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/205242","number":205242,"mergeCommit":{"message":"[APM
Fix missing error.culprit (#205242)\n\nfixes
[#205240](https://github.com/elastic/kibana/issues/205240)\r\n\r\n##
Summary\r\n\r\nFixes the issue where `error.culprit` is always
missing.\r\n\r\n<img width=\"800\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/9dad6797-dd40-4206-b508-daf8d69d653d\"\r\n/>\r\n\r\n<img
width=\"800\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/6a0ac25a-bc4b-4a26-b126-86eecbe7e769\"\r\n/>\r\n\r\n\r\n\r\n###
How to test \r\n\r\n- Run `simple_trace` synthtrace scenario\r\n- Run
the request below\r\n<details>\r\n<summary>POST
logs-apm.error-default/_doc</summary>\r\n\r\n```\r\nPOST
logs-apm.error-default/_doc\r\n{\r\n\r\n \"container\": {\r\n \"id\":
\"80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359\"\r\n
},\r\n \"kubernetes\": {\r\n \"node\": {\r\n \"name\":
\"ip-192-168-55-136.ec2.internal\"\r\n },\r\n \"pod\": {\r\n \"uid\":
\"9bbfe3dc-dace-4638-b8f5-aa6ccf435cbe\",\r\n \"name\":
\"otel-demo-green-3-cartservice-5d5ccc86dd-gcxtr\"\r\n },\r\n
\"namespace\": \"otel-demo-green-3\"\r\n },\r\n \"parent\": {\r\n
\"id\": \"f39639f56a540e0e\"\r\n },\r\n \"agent\": {\r\n \"name\":
\"opentelemetry/dotnet\",\r\n \"version\": \"1.6.0\"\r\n },\r\n
\"source\": {\r\n \"port\": 35136,\r\n \"ip\":
\"::ffff:192.168.50.132\"\r\n },\r\n \"error\": {\r\n \"exception\":
[\r\n {\r\n \"handled\": true,\r\n \"message\": \"Exception (plain,
.NET) without culprit\",\r\n \"type\": \"Grpc.Core.RpcException\"\r\n
}\r\n ],\r\n \"culprit\": \"culprit msg\", // remove to test the N/A
scenario\r\n \"id\": \"60674e9b23c8b06d83889e63ffe82c1c\", //change the
id on every run\r\n \"stack_trace\": \"Grpc.Core.RpcException:
Status(StatusCode=\\\"FailedPrecondition\\\", Detail=\\\"Can't access
cart storage. System.ApplicationException: Wasn't able to connect to
redis\\n at cartservice.cartstore.RedisCartStore.EnsureRedisConnected()
in /usr/src/app/src/cartstore/RedisCartStore.cs:line 79\\n at
cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in
/usr/src/app/src/cartstore/RedisCartStore.cs:line 157\\\")\\n at
cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in
/usr/src/app/src/cartstore/RedisCartStore.cs:line 166\\n at
cartservice.services.CartService.EmptyCart(EmptyCartRequest request,
ServerCallContext context) in
/usr/src/app/src/services/CartService.cs:line 65\",\r\n
\"grouping_key\": \"6082c58afd7fa6838ebc27eef6e04829\"\r\n },\r\n
\"message\": \"Exception (plain, .NET)\",\r\n \"url\": {\r\n \"path\":
\"/oteldemo.CartService/EmptyCart\",\r\n \"original\":
\"http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart\",\r\n
\"scheme\": \"http\",\r\n \"port\": 8080,\r\n \"domain\":
\"otel-demo-green-3-cartservice\",\r\n \"full\":
\"http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart\"\r\n
},\r\n \"labels\": {\r\n \"k8s_pod_ip\": \"192.168.62.174\",\r\n
\"k8s_pod_start_time\": \"2024-12-17T02:55:54Z\",\r\n
\"service_namespace\": \"opentelemetry-demo\",\r\n
\"k8s_deployment_name\": \"otel-demo-green-3-cartservice\"\r\n },\r\n
\"observer\": {\r\n \"hostname\": \"450fbf7ab701\",\r\n \"type\":
\"apm-server\",\r\n \"version\": \"8.13.2\"\r\n },\r\n \"trace\": {\r\n
\"id\": \"699f6d1039fc873191a1c983790db3a2\"\r\n },\r\n \"@timestamp\":
\"2024-12-30T10:30:00.000Z\",\r\n \"data_stream\": {\r\n \"namespace\":
\"default\",\r\n \"type\": \"logs\",\r\n \"dataset\": \"apm.error\"\r\n
},\r\n \"service\": {\r\n \"node\": {\r\n \"name\":
\"80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359\"\r\n
},\r\n \"framework\": {\r\n \"name\": \"Microsoft.AspNetCore\"\r\n
},\r\n \"name\": \"synth-node-0\",\r\n \"language\": {\r\n \"name\":
\"dotnet\"\r\n }\r\n },\r\n \"host\": {\r\n \"hostname\":
\"ip-192-168-55-136.ec2.internal\",\r\n \"name\":
\"ip-192-168-55-136.ec2.internal\"\r\n },\r\n \"client\": {\r\n
\"port\": 35136,\r\n \"ip\": \"::ffff:192.168.50.132\"\r\n },\r\n
\"http\": {\r\n \"request\": {\r\n \"method\": \"POST\"\r\n },\r\n
\"response\": {\r\n \"status_code\": 200\r\n },\r\n \"version\":
\"2\"\r\n },\r\n \"event\": {\r\n \"agent_id_status\": \"missing\",\r\n
\"ingested\": \"2024-12-18T18:04:22Z\"\r\n },\r\n \"transaction\": {\r\n
\"id\": \"f39639f56a540e0e\",\r\n \"type\": \"request\",\r\n
\"sampled\": true\r\n },\r\n \"user_agent\": {\r\n \"original\":
\"grpc-go/1.59.0\",\r\n \"name\": \"Other\",\r\n \"device\": {\r\n
\"name\": \"Other\"\r\n }\r\n },\r\n \"span\": {\r\n \"id\":
\"f39639f56a540e0e\"\r\n },\r\n \"timestamp\": {\r\n \"us\":
1704892605838000\r\n }\r\n
}\r\n```\r\n</details>","sha":"12608c3659a2aaea702c1d341bd0fab30f968056"}}]}]
BACKPORT-->

Co-authored-by: Carlos Crespo <[email protected]>
kibanamachine added a commit that referenced this pull request Dec 30, 2024
# Backport

This will backport the following commits from `main` to `8.x`:
- [[APM Fix missing error.culprit
(#205242)](#205242)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Carlos
Crespo","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-12-30T12:50:48Z","message":"[APM
Fix missing error.culprit (#205242)\n\nfixes
[#205240](https://github.com/elastic/kibana/issues/205240)\r\n\r\n##
Summary\r\n\r\nFixes the issue where `error.culprit` is always
missing.\r\n\r\n<img width=\"800\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/9dad6797-dd40-4206-b508-daf8d69d653d\"\r\n/>\r\n\r\n<img
width=\"800\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/6a0ac25a-bc4b-4a26-b126-86eecbe7e769\"\r\n/>\r\n\r\n\r\n\r\n###
How to test \r\n\r\n- Run `simple_trace` synthtrace scenario\r\n- Run
the request below\r\n<details>\r\n<summary>POST
logs-apm.error-default/_doc</summary>\r\n\r\n```\r\nPOST
logs-apm.error-default/_doc\r\n{\r\n\r\n \"container\": {\r\n \"id\":
\"80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359\"\r\n
},\r\n \"kubernetes\": {\r\n \"node\": {\r\n \"name\":
\"ip-192-168-55-136.ec2.internal\"\r\n },\r\n \"pod\": {\r\n \"uid\":
\"9bbfe3dc-dace-4638-b8f5-aa6ccf435cbe\",\r\n \"name\":
\"otel-demo-green-3-cartservice-5d5ccc86dd-gcxtr\"\r\n },\r\n
\"namespace\": \"otel-demo-green-3\"\r\n },\r\n \"parent\": {\r\n
\"id\": \"f39639f56a540e0e\"\r\n },\r\n \"agent\": {\r\n \"name\":
\"opentelemetry/dotnet\",\r\n \"version\": \"1.6.0\"\r\n },\r\n
\"source\": {\r\n \"port\": 35136,\r\n \"ip\":
\"::ffff:192.168.50.132\"\r\n },\r\n \"error\": {\r\n \"exception\":
[\r\n {\r\n \"handled\": true,\r\n \"message\": \"Exception (plain,
.NET) without culprit\",\r\n \"type\": \"Grpc.Core.RpcException\"\r\n
}\r\n ],\r\n \"culprit\": \"culprit msg\", // remove to test the N/A
scenario\r\n \"id\": \"60674e9b23c8b06d83889e63ffe82c1c\", //change the
id on every run\r\n \"stack_trace\": \"Grpc.Core.RpcException:
Status(StatusCode=\\\"FailedPrecondition\\\", Detail=\\\"Can't access
cart storage. System.ApplicationException: Wasn't able to connect to
redis\\n at cartservice.cartstore.RedisCartStore.EnsureRedisConnected()
in /usr/src/app/src/cartstore/RedisCartStore.cs:line 79\\n at
cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in
/usr/src/app/src/cartstore/RedisCartStore.cs:line 157\\\")\\n at
cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in
/usr/src/app/src/cartstore/RedisCartStore.cs:line 166\\n at
cartservice.services.CartService.EmptyCart(EmptyCartRequest request,
ServerCallContext context) in
/usr/src/app/src/services/CartService.cs:line 65\",\r\n
\"grouping_key\": \"6082c58afd7fa6838ebc27eef6e04829\"\r\n },\r\n
\"message\": \"Exception (plain, .NET)\",\r\n \"url\": {\r\n \"path\":
\"/oteldemo.CartService/EmptyCart\",\r\n \"original\":
\"http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart\",\r\n
\"scheme\": \"http\",\r\n \"port\": 8080,\r\n \"domain\":
\"otel-demo-green-3-cartservice\",\r\n \"full\":
\"http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart\"\r\n
},\r\n \"labels\": {\r\n \"k8s_pod_ip\": \"192.168.62.174\",\r\n
\"k8s_pod_start_time\": \"2024-12-17T02:55:54Z\",\r\n
\"service_namespace\": \"opentelemetry-demo\",\r\n
\"k8s_deployment_name\": \"otel-demo-green-3-cartservice\"\r\n },\r\n
\"observer\": {\r\n \"hostname\": \"450fbf7ab701\",\r\n \"type\":
\"apm-server\",\r\n \"version\": \"8.13.2\"\r\n },\r\n \"trace\": {\r\n
\"id\": \"699f6d1039fc873191a1c983790db3a2\"\r\n },\r\n \"@timestamp\":
\"2024-12-30T10:30:00.000Z\",\r\n \"data_stream\": {\r\n \"namespace\":
\"default\",\r\n \"type\": \"logs\",\r\n \"dataset\": \"apm.error\"\r\n
},\r\n \"service\": {\r\n \"node\": {\r\n \"name\":
\"80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359\"\r\n
},\r\n \"framework\": {\r\n \"name\": \"Microsoft.AspNetCore\"\r\n
},\r\n \"name\": \"synth-node-0\",\r\n \"language\": {\r\n \"name\":
\"dotnet\"\r\n }\r\n },\r\n \"host\": {\r\n \"hostname\":
\"ip-192-168-55-136.ec2.internal\",\r\n \"name\":
\"ip-192-168-55-136.ec2.internal\"\r\n },\r\n \"client\": {\r\n
\"port\": 35136,\r\n \"ip\": \"::ffff:192.168.50.132\"\r\n },\r\n
\"http\": {\r\n \"request\": {\r\n \"method\": \"POST\"\r\n },\r\n
\"response\": {\r\n \"status_code\": 200\r\n },\r\n \"version\":
\"2\"\r\n },\r\n \"event\": {\r\n \"agent_id_status\": \"missing\",\r\n
\"ingested\": \"2024-12-18T18:04:22Z\"\r\n },\r\n \"transaction\": {\r\n
\"id\": \"f39639f56a540e0e\",\r\n \"type\": \"request\",\r\n
\"sampled\": true\r\n },\r\n \"user_agent\": {\r\n \"original\":
\"grpc-go/1.59.0\",\r\n \"name\": \"Other\",\r\n \"device\": {\r\n
\"name\": \"Other\"\r\n }\r\n },\r\n \"span\": {\r\n \"id\":
\"f39639f56a540e0e\"\r\n },\r\n \"timestamp\": {\r\n \"us\":
1704892605838000\r\n }\r\n
}\r\n```\r\n</details>","sha":"12608c3659a2aaea702c1d341bd0fab30f968056","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","backport:prev-major","Team:obs-ux-infra_services"],"title":"[APM
Fix missing
error.culprit","number":205242,"url":"https://github.com/elastic/kibana/pull/205242","mergeCommit":{"message":"[APM
Fix missing error.culprit (#205242)\n\nfixes
[#205240](https://github.com/elastic/kibana/issues/205240)\r\n\r\n##
Summary\r\n\r\nFixes the issue where `error.culprit` is always
missing.\r\n\r\n<img width=\"800\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/9dad6797-dd40-4206-b508-daf8d69d653d\"\r\n/>\r\n\r\n<img
width=\"800\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/6a0ac25a-bc4b-4a26-b126-86eecbe7e769\"\r\n/>\r\n\r\n\r\n\r\n###
How to test \r\n\r\n- Run `simple_trace` synthtrace scenario\r\n- Run
the request below\r\n<details>\r\n<summary>POST
logs-apm.error-default/_doc</summary>\r\n\r\n```\r\nPOST
logs-apm.error-default/_doc\r\n{\r\n\r\n \"container\": {\r\n \"id\":
\"80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359\"\r\n
},\r\n \"kubernetes\": {\r\n \"node\": {\r\n \"name\":
\"ip-192-168-55-136.ec2.internal\"\r\n },\r\n \"pod\": {\r\n \"uid\":
\"9bbfe3dc-dace-4638-b8f5-aa6ccf435cbe\",\r\n \"name\":
\"otel-demo-green-3-cartservice-5d5ccc86dd-gcxtr\"\r\n },\r\n
\"namespace\": \"otel-demo-green-3\"\r\n },\r\n \"parent\": {\r\n
\"id\": \"f39639f56a540e0e\"\r\n },\r\n \"agent\": {\r\n \"name\":
\"opentelemetry/dotnet\",\r\n \"version\": \"1.6.0\"\r\n },\r\n
\"source\": {\r\n \"port\": 35136,\r\n \"ip\":
\"::ffff:192.168.50.132\"\r\n },\r\n \"error\": {\r\n \"exception\":
[\r\n {\r\n \"handled\": true,\r\n \"message\": \"Exception (plain,
.NET) without culprit\",\r\n \"type\": \"Grpc.Core.RpcException\"\r\n
}\r\n ],\r\n \"culprit\": \"culprit msg\", // remove to test the N/A
scenario\r\n \"id\": \"60674e9b23c8b06d83889e63ffe82c1c\", //change the
id on every run\r\n \"stack_trace\": \"Grpc.Core.RpcException:
Status(StatusCode=\\\"FailedPrecondition\\\", Detail=\\\"Can't access
cart storage. System.ApplicationException: Wasn't able to connect to
redis\\n at cartservice.cartstore.RedisCartStore.EnsureRedisConnected()
in /usr/src/app/src/cartstore/RedisCartStore.cs:line 79\\n at
cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in
/usr/src/app/src/cartstore/RedisCartStore.cs:line 157\\\")\\n at
cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in
/usr/src/app/src/cartstore/RedisCartStore.cs:line 166\\n at
cartservice.services.CartService.EmptyCart(EmptyCartRequest request,
ServerCallContext context) in
/usr/src/app/src/services/CartService.cs:line 65\",\r\n
\"grouping_key\": \"6082c58afd7fa6838ebc27eef6e04829\"\r\n },\r\n
\"message\": \"Exception (plain, .NET)\",\r\n \"url\": {\r\n \"path\":
\"/oteldemo.CartService/EmptyCart\",\r\n \"original\":
\"http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart\",\r\n
\"scheme\": \"http\",\r\n \"port\": 8080,\r\n \"domain\":
\"otel-demo-green-3-cartservice\",\r\n \"full\":
\"http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart\"\r\n
},\r\n \"labels\": {\r\n \"k8s_pod_ip\": \"192.168.62.174\",\r\n
\"k8s_pod_start_time\": \"2024-12-17T02:55:54Z\",\r\n
\"service_namespace\": \"opentelemetry-demo\",\r\n
\"k8s_deployment_name\": \"otel-demo-green-3-cartservice\"\r\n },\r\n
\"observer\": {\r\n \"hostname\": \"450fbf7ab701\",\r\n \"type\":
\"apm-server\",\r\n \"version\": \"8.13.2\"\r\n },\r\n \"trace\": {\r\n
\"id\": \"699f6d1039fc873191a1c983790db3a2\"\r\n },\r\n \"@timestamp\":
\"2024-12-30T10:30:00.000Z\",\r\n \"data_stream\": {\r\n \"namespace\":
\"default\",\r\n \"type\": \"logs\",\r\n \"dataset\": \"apm.error\"\r\n
},\r\n \"service\": {\r\n \"node\": {\r\n \"name\":
\"80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359\"\r\n
},\r\n \"framework\": {\r\n \"name\": \"Microsoft.AspNetCore\"\r\n
},\r\n \"name\": \"synth-node-0\",\r\n \"language\": {\r\n \"name\":
\"dotnet\"\r\n }\r\n },\r\n \"host\": {\r\n \"hostname\":
\"ip-192-168-55-136.ec2.internal\",\r\n \"name\":
\"ip-192-168-55-136.ec2.internal\"\r\n },\r\n \"client\": {\r\n
\"port\": 35136,\r\n \"ip\": \"::ffff:192.168.50.132\"\r\n },\r\n
\"http\": {\r\n \"request\": {\r\n \"method\": \"POST\"\r\n },\r\n
\"response\": {\r\n \"status_code\": 200\r\n },\r\n \"version\":
\"2\"\r\n },\r\n \"event\": {\r\n \"agent_id_status\": \"missing\",\r\n
\"ingested\": \"2024-12-18T18:04:22Z\"\r\n },\r\n \"transaction\": {\r\n
\"id\": \"f39639f56a540e0e\",\r\n \"type\": \"request\",\r\n
\"sampled\": true\r\n },\r\n \"user_agent\": {\r\n \"original\":
\"grpc-go/1.59.0\",\r\n \"name\": \"Other\",\r\n \"device\": {\r\n
\"name\": \"Other\"\r\n }\r\n },\r\n \"span\": {\r\n \"id\":
\"f39639f56a540e0e\"\r\n },\r\n \"timestamp\": {\r\n \"us\":
1704892605838000\r\n }\r\n
}\r\n```\r\n</details>","sha":"12608c3659a2aaea702c1d341bd0fab30f968056"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/205242","number":205242,"mergeCommit":{"message":"[APM
Fix missing error.culprit (#205242)\n\nfixes
[#205240](https://github.com/elastic/kibana/issues/205240)\r\n\r\n##
Summary\r\n\r\nFixes the issue where `error.culprit` is always
missing.\r\n\r\n<img width=\"800\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/9dad6797-dd40-4206-b508-daf8d69d653d\"\r\n/>\r\n\r\n<img
width=\"800\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/6a0ac25a-bc4b-4a26-b126-86eecbe7e769\"\r\n/>\r\n\r\n\r\n\r\n###
How to test \r\n\r\n- Run `simple_trace` synthtrace scenario\r\n- Run
the request below\r\n<details>\r\n<summary>POST
logs-apm.error-default/_doc</summary>\r\n\r\n```\r\nPOST
logs-apm.error-default/_doc\r\n{\r\n\r\n \"container\": {\r\n \"id\":
\"80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359\"\r\n
},\r\n \"kubernetes\": {\r\n \"node\": {\r\n \"name\":
\"ip-192-168-55-136.ec2.internal\"\r\n },\r\n \"pod\": {\r\n \"uid\":
\"9bbfe3dc-dace-4638-b8f5-aa6ccf435cbe\",\r\n \"name\":
\"otel-demo-green-3-cartservice-5d5ccc86dd-gcxtr\"\r\n },\r\n
\"namespace\": \"otel-demo-green-3\"\r\n },\r\n \"parent\": {\r\n
\"id\": \"f39639f56a540e0e\"\r\n },\r\n \"agent\": {\r\n \"name\":
\"opentelemetry/dotnet\",\r\n \"version\": \"1.6.0\"\r\n },\r\n
\"source\": {\r\n \"port\": 35136,\r\n \"ip\":
\"::ffff:192.168.50.132\"\r\n },\r\n \"error\": {\r\n \"exception\":
[\r\n {\r\n \"handled\": true,\r\n \"message\": \"Exception (plain,
.NET) without culprit\",\r\n \"type\": \"Grpc.Core.RpcException\"\r\n
}\r\n ],\r\n \"culprit\": \"culprit msg\", // remove to test the N/A
scenario\r\n \"id\": \"60674e9b23c8b06d83889e63ffe82c1c\", //change the
id on every run\r\n \"stack_trace\": \"Grpc.Core.RpcException:
Status(StatusCode=\\\"FailedPrecondition\\\", Detail=\\\"Can't access
cart storage. System.ApplicationException: Wasn't able to connect to
redis\\n at cartservice.cartstore.RedisCartStore.EnsureRedisConnected()
in /usr/src/app/src/cartstore/RedisCartStore.cs:line 79\\n at
cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in
/usr/src/app/src/cartstore/RedisCartStore.cs:line 157\\\")\\n at
cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in
/usr/src/app/src/cartstore/RedisCartStore.cs:line 166\\n at
cartservice.services.CartService.EmptyCart(EmptyCartRequest request,
ServerCallContext context) in
/usr/src/app/src/services/CartService.cs:line 65\",\r\n
\"grouping_key\": \"6082c58afd7fa6838ebc27eef6e04829\"\r\n },\r\n
\"message\": \"Exception (plain, .NET)\",\r\n \"url\": {\r\n \"path\":
\"/oteldemo.CartService/EmptyCart\",\r\n \"original\":
\"http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart\",\r\n
\"scheme\": \"http\",\r\n \"port\": 8080,\r\n \"domain\":
\"otel-demo-green-3-cartservice\",\r\n \"full\":
\"http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart\"\r\n
},\r\n \"labels\": {\r\n \"k8s_pod_ip\": \"192.168.62.174\",\r\n
\"k8s_pod_start_time\": \"2024-12-17T02:55:54Z\",\r\n
\"service_namespace\": \"opentelemetry-demo\",\r\n
\"k8s_deployment_name\": \"otel-demo-green-3-cartservice\"\r\n },\r\n
\"observer\": {\r\n \"hostname\": \"450fbf7ab701\",\r\n \"type\":
\"apm-server\",\r\n \"version\": \"8.13.2\"\r\n },\r\n \"trace\": {\r\n
\"id\": \"699f6d1039fc873191a1c983790db3a2\"\r\n },\r\n \"@timestamp\":
\"2024-12-30T10:30:00.000Z\",\r\n \"data_stream\": {\r\n \"namespace\":
\"default\",\r\n \"type\": \"logs\",\r\n \"dataset\": \"apm.error\"\r\n
},\r\n \"service\": {\r\n \"node\": {\r\n \"name\":
\"80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359\"\r\n
},\r\n \"framework\": {\r\n \"name\": \"Microsoft.AspNetCore\"\r\n
},\r\n \"name\": \"synth-node-0\",\r\n \"language\": {\r\n \"name\":
\"dotnet\"\r\n }\r\n },\r\n \"host\": {\r\n \"hostname\":
\"ip-192-168-55-136.ec2.internal\",\r\n \"name\":
\"ip-192-168-55-136.ec2.internal\"\r\n },\r\n \"client\": {\r\n
\"port\": 35136,\r\n \"ip\": \"::ffff:192.168.50.132\"\r\n },\r\n
\"http\": {\r\n \"request\": {\r\n \"method\": \"POST\"\r\n },\r\n
\"response\": {\r\n \"status_code\": 200\r\n },\r\n \"version\":
\"2\"\r\n },\r\n \"event\": {\r\n \"agent_id_status\": \"missing\",\r\n
\"ingested\": \"2024-12-18T18:04:22Z\"\r\n },\r\n \"transaction\": {\r\n
\"id\": \"f39639f56a540e0e\",\r\n \"type\": \"request\",\r\n
\"sampled\": true\r\n },\r\n \"user_agent\": {\r\n \"original\":
\"grpc-go/1.59.0\",\r\n \"name\": \"Other\",\r\n \"device\": {\r\n
\"name\": \"Other\"\r\n }\r\n },\r\n \"span\": {\r\n \"id\":
\"f39639f56a540e0e\"\r\n },\r\n \"timestamp\": {\r\n \"us\":
1704892605838000\r\n }\r\n
}\r\n```\r\n</details>","sha":"12608c3659a2aaea702c1d341bd0fab30f968056"}}]}]
BACKPORT-->

Co-authored-by: Carlos Crespo <[email protected]>
crespocarlos added a commit that referenced this pull request Dec 30, 2024
# Backport

This will backport the following commits from `main` to `8.16`:
- [[APM Fix missing error.culprit
(#205242)](#205242)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Carlos
Crespo","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-12-30T12:50:48Z","message":"[APM
Fix missing error.culprit (#205242)\n\nfixes
[#205240](https://github.com/elastic/kibana/issues/205240)\r\n\r\n##
Summary\r\n\r\nFixes the issue where `error.culprit` is always
missing.\r\n\r\n<img width=\"800\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/9dad6797-dd40-4206-b508-daf8d69d653d\"\r\n/>\r\n\r\n<img
width=\"800\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/6a0ac25a-bc4b-4a26-b126-86eecbe7e769\"\r\n/>\r\n\r\n\r\n\r\n###
How to test \r\n\r\n- Run `simple_trace` synthtrace scenario\r\n- Run
the request below\r\n<details>\r\n<summary>POST
logs-apm.error-default/_doc</summary>\r\n\r\n```\r\nPOST
logs-apm.error-default/_doc\r\n{\r\n\r\n \"container\": {\r\n \"id\":
\"80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359\"\r\n
},\r\n \"kubernetes\": {\r\n \"node\": {\r\n \"name\":
\"ip-192-168-55-136.ec2.internal\"\r\n },\r\n \"pod\": {\r\n \"uid\":
\"9bbfe3dc-dace-4638-b8f5-aa6ccf435cbe\",\r\n \"name\":
\"otel-demo-green-3-cartservice-5d5ccc86dd-gcxtr\"\r\n },\r\n
\"namespace\": \"otel-demo-green-3\"\r\n },\r\n \"parent\": {\r\n
\"id\": \"f39639f56a540e0e\"\r\n },\r\n \"agent\": {\r\n \"name\":
\"opentelemetry/dotnet\",\r\n \"version\": \"1.6.0\"\r\n },\r\n
\"source\": {\r\n \"port\": 35136,\r\n \"ip\":
\"::ffff:192.168.50.132\"\r\n },\r\n \"error\": {\r\n \"exception\":
[\r\n {\r\n \"handled\": true,\r\n \"message\": \"Exception (plain,
.NET) without culprit\",\r\n \"type\": \"Grpc.Core.RpcException\"\r\n
}\r\n ],\r\n \"culprit\": \"culprit msg\", // remove to test the N/A
scenario\r\n \"id\": \"60674e9b23c8b06d83889e63ffe82c1c\", //change the
id on every run\r\n \"stack_trace\": \"Grpc.Core.RpcException:
Status(StatusCode=\\\"FailedPrecondition\\\", Detail=\\\"Can't access
cart storage. System.ApplicationException: Wasn't able to connect to
redis\\n at cartservice.cartstore.RedisCartStore.EnsureRedisConnected()
in /usr/src/app/src/cartstore/RedisCartStore.cs:line 79\\n at
cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in
/usr/src/app/src/cartstore/RedisCartStore.cs:line 157\\\")\\n at
cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in
/usr/src/app/src/cartstore/RedisCartStore.cs:line 166\\n at
cartservice.services.CartService.EmptyCart(EmptyCartRequest request,
ServerCallContext context) in
/usr/src/app/src/services/CartService.cs:line 65\",\r\n
\"grouping_key\": \"6082c58afd7fa6838ebc27eef6e04829\"\r\n },\r\n
\"message\": \"Exception (plain, .NET)\",\r\n \"url\": {\r\n \"path\":
\"/oteldemo.CartService/EmptyCart\",\r\n \"original\":
\"http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart\",\r\n
\"scheme\": \"http\",\r\n \"port\": 8080,\r\n \"domain\":
\"otel-demo-green-3-cartservice\",\r\n \"full\":
\"http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart\"\r\n
},\r\n \"labels\": {\r\n \"k8s_pod_ip\": \"192.168.62.174\",\r\n
\"k8s_pod_start_time\": \"2024-12-17T02:55:54Z\",\r\n
\"service_namespace\": \"opentelemetry-demo\",\r\n
\"k8s_deployment_name\": \"otel-demo-green-3-cartservice\"\r\n },\r\n
\"observer\": {\r\n \"hostname\": \"450fbf7ab701\",\r\n \"type\":
\"apm-server\",\r\n \"version\": \"8.13.2\"\r\n },\r\n \"trace\": {\r\n
\"id\": \"699f6d1039fc873191a1c983790db3a2\"\r\n },\r\n \"@timestamp\":
\"2024-12-30T10:30:00.000Z\",\r\n \"data_stream\": {\r\n \"namespace\":
\"default\",\r\n \"type\": \"logs\",\r\n \"dataset\": \"apm.error\"\r\n
},\r\n \"service\": {\r\n \"node\": {\r\n \"name\":
\"80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359\"\r\n
},\r\n \"framework\": {\r\n \"name\": \"Microsoft.AspNetCore\"\r\n
},\r\n \"name\": \"synth-node-0\",\r\n \"language\": {\r\n \"name\":
\"dotnet\"\r\n }\r\n },\r\n \"host\": {\r\n \"hostname\":
\"ip-192-168-55-136.ec2.internal\",\r\n \"name\":
\"ip-192-168-55-136.ec2.internal\"\r\n },\r\n \"client\": {\r\n
\"port\": 35136,\r\n \"ip\": \"::ffff:192.168.50.132\"\r\n },\r\n
\"http\": {\r\n \"request\": {\r\n \"method\": \"POST\"\r\n },\r\n
\"response\": {\r\n \"status_code\": 200\r\n },\r\n \"version\":
\"2\"\r\n },\r\n \"event\": {\r\n \"agent_id_status\": \"missing\",\r\n
\"ingested\": \"2024-12-18T18:04:22Z\"\r\n },\r\n \"transaction\": {\r\n
\"id\": \"f39639f56a540e0e\",\r\n \"type\": \"request\",\r\n
\"sampled\": true\r\n },\r\n \"user_agent\": {\r\n \"original\":
\"grpc-go/1.59.0\",\r\n \"name\": \"Other\",\r\n \"device\": {\r\n
\"name\": \"Other\"\r\n }\r\n },\r\n \"span\": {\r\n \"id\":
\"f39639f56a540e0e\"\r\n },\r\n \"timestamp\": {\r\n \"us\":
1704892605838000\r\n }\r\n
}\r\n```\r\n</details>","sha":"12608c3659a2aaea702c1d341bd0fab30f968056","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","backport:prev-major","Team:obs-ux-infra_services"],"number":205242,"url":"https://github.com/elastic/kibana/pull/205242","mergeCommit":{"message":"[APM
Fix missing error.culprit (#205242)\n\nfixes
[#205240](https://github.com/elastic/kibana/issues/205240)\r\n\r\n##
Summary\r\n\r\nFixes the issue where `error.culprit` is always
missing.\r\n\r\n<img width=\"800\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/9dad6797-dd40-4206-b508-daf8d69d653d\"\r\n/>\r\n\r\n<img
width=\"800\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/6a0ac25a-bc4b-4a26-b126-86eecbe7e769\"\r\n/>\r\n\r\n\r\n\r\n###
How to test \r\n\r\n- Run `simple_trace` synthtrace scenario\r\n- Run
the request below\r\n<details>\r\n<summary>POST
logs-apm.error-default/_doc</summary>\r\n\r\n```\r\nPOST
logs-apm.error-default/_doc\r\n{\r\n\r\n \"container\": {\r\n \"id\":
\"80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359\"\r\n
},\r\n \"kubernetes\": {\r\n \"node\": {\r\n \"name\":
\"ip-192-168-55-136.ec2.internal\"\r\n },\r\n \"pod\": {\r\n \"uid\":
\"9bbfe3dc-dace-4638-b8f5-aa6ccf435cbe\",\r\n \"name\":
\"otel-demo-green-3-cartservice-5d5ccc86dd-gcxtr\"\r\n },\r\n
\"namespace\": \"otel-demo-green-3\"\r\n },\r\n \"parent\": {\r\n
\"id\": \"f39639f56a540e0e\"\r\n },\r\n \"agent\": {\r\n \"name\":
\"opentelemetry/dotnet\",\r\n \"version\": \"1.6.0\"\r\n },\r\n
\"source\": {\r\n \"port\": 35136,\r\n \"ip\":
\"::ffff:192.168.50.132\"\r\n },\r\n \"error\": {\r\n \"exception\":
[\r\n {\r\n \"handled\": true,\r\n \"message\": \"Exception (plain,
.NET) without culprit\",\r\n \"type\": \"Grpc.Core.RpcException\"\r\n
}\r\n ],\r\n \"culprit\": \"culprit msg\", // remove to test the N/A
scenario\r\n \"id\": \"60674e9b23c8b06d83889e63ffe82c1c\", //change the
id on every run\r\n \"stack_trace\": \"Grpc.Core.RpcException:
Status(StatusCode=\\\"FailedPrecondition\\\", Detail=\\\"Can't access
cart storage. System.ApplicationException: Wasn't able to connect to
redis\\n at cartservice.cartstore.RedisCartStore.EnsureRedisConnected()
in /usr/src/app/src/cartstore/RedisCartStore.cs:line 79\\n at
cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in
/usr/src/app/src/cartstore/RedisCartStore.cs:line 157\\\")\\n at
cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in
/usr/src/app/src/cartstore/RedisCartStore.cs:line 166\\n at
cartservice.services.CartService.EmptyCart(EmptyCartRequest request,
ServerCallContext context) in
/usr/src/app/src/services/CartService.cs:line 65\",\r\n
\"grouping_key\": \"6082c58afd7fa6838ebc27eef6e04829\"\r\n },\r\n
\"message\": \"Exception (plain, .NET)\",\r\n \"url\": {\r\n \"path\":
\"/oteldemo.CartService/EmptyCart\",\r\n \"original\":
\"http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart\",\r\n
\"scheme\": \"http\",\r\n \"port\": 8080,\r\n \"domain\":
\"otel-demo-green-3-cartservice\",\r\n \"full\":
\"http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart\"\r\n
},\r\n \"labels\": {\r\n \"k8s_pod_ip\": \"192.168.62.174\",\r\n
\"k8s_pod_start_time\": \"2024-12-17T02:55:54Z\",\r\n
\"service_namespace\": \"opentelemetry-demo\",\r\n
\"k8s_deployment_name\": \"otel-demo-green-3-cartservice\"\r\n },\r\n
\"observer\": {\r\n \"hostname\": \"450fbf7ab701\",\r\n \"type\":
\"apm-server\",\r\n \"version\": \"8.13.2\"\r\n },\r\n \"trace\": {\r\n
\"id\": \"699f6d1039fc873191a1c983790db3a2\"\r\n },\r\n \"@timestamp\":
\"2024-12-30T10:30:00.000Z\",\r\n \"data_stream\": {\r\n \"namespace\":
\"default\",\r\n \"type\": \"logs\",\r\n \"dataset\": \"apm.error\"\r\n
},\r\n \"service\": {\r\n \"node\": {\r\n \"name\":
\"80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359\"\r\n
},\r\n \"framework\": {\r\n \"name\": \"Microsoft.AspNetCore\"\r\n
},\r\n \"name\": \"synth-node-0\",\r\n \"language\": {\r\n \"name\":
\"dotnet\"\r\n }\r\n },\r\n \"host\": {\r\n \"hostname\":
\"ip-192-168-55-136.ec2.internal\",\r\n \"name\":
\"ip-192-168-55-136.ec2.internal\"\r\n },\r\n \"client\": {\r\n
\"port\": 35136,\r\n \"ip\": \"::ffff:192.168.50.132\"\r\n },\r\n
\"http\": {\r\n \"request\": {\r\n \"method\": \"POST\"\r\n },\r\n
\"response\": {\r\n \"status_code\": 200\r\n },\r\n \"version\":
\"2\"\r\n },\r\n \"event\": {\r\n \"agent_id_status\": \"missing\",\r\n
\"ingested\": \"2024-12-18T18:04:22Z\"\r\n },\r\n \"transaction\": {\r\n
\"id\": \"f39639f56a540e0e\",\r\n \"type\": \"request\",\r\n
\"sampled\": true\r\n },\r\n \"user_agent\": {\r\n \"original\":
\"grpc-go/1.59.0\",\r\n \"name\": \"Other\",\r\n \"device\": {\r\n
\"name\": \"Other\"\r\n }\r\n },\r\n \"span\": {\r\n \"id\":
\"f39639f56a540e0e\"\r\n },\r\n \"timestamp\": {\r\n \"us\":
1704892605838000\r\n }\r\n
}\r\n```\r\n</details>","sha":"12608c3659a2aaea702c1d341bd0fab30f968056"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/205242","number":205242,"mergeCommit":{"message":"[APM
Fix missing error.culprit (#205242)\n\nfixes
[#205240](https://github.com/elastic/kibana/issues/205240)\r\n\r\n##
Summary\r\n\r\nFixes the issue where `error.culprit` is always
missing.\r\n\r\n<img width=\"800\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/9dad6797-dd40-4206-b508-daf8d69d653d\"\r\n/>\r\n\r\n<img
width=\"800\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/6a0ac25a-bc4b-4a26-b126-86eecbe7e769\"\r\n/>\r\n\r\n\r\n\r\n###
How to test \r\n\r\n- Run `simple_trace` synthtrace scenario\r\n- Run
the request below\r\n<details>\r\n<summary>POST
logs-apm.error-default/_doc</summary>\r\n\r\n```\r\nPOST
logs-apm.error-default/_doc\r\n{\r\n\r\n \"container\": {\r\n \"id\":
\"80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359\"\r\n
},\r\n \"kubernetes\": {\r\n \"node\": {\r\n \"name\":
\"ip-192-168-55-136.ec2.internal\"\r\n },\r\n \"pod\": {\r\n \"uid\":
\"9bbfe3dc-dace-4638-b8f5-aa6ccf435cbe\",\r\n \"name\":
\"otel-demo-green-3-cartservice-5d5ccc86dd-gcxtr\"\r\n },\r\n
\"namespace\": \"otel-demo-green-3\"\r\n },\r\n \"parent\": {\r\n
\"id\": \"f39639f56a540e0e\"\r\n },\r\n \"agent\": {\r\n \"name\":
\"opentelemetry/dotnet\",\r\n \"version\": \"1.6.0\"\r\n },\r\n
\"source\": {\r\n \"port\": 35136,\r\n \"ip\":
\"::ffff:192.168.50.132\"\r\n },\r\n \"error\": {\r\n \"exception\":
[\r\n {\r\n \"handled\": true,\r\n \"message\": \"Exception (plain,
.NET) without culprit\",\r\n \"type\": \"Grpc.Core.RpcException\"\r\n
}\r\n ],\r\n \"culprit\": \"culprit msg\", // remove to test the N/A
scenario\r\n \"id\": \"60674e9b23c8b06d83889e63ffe82c1c\", //change the
id on every run\r\n \"stack_trace\": \"Grpc.Core.RpcException:
Status(StatusCode=\\\"FailedPrecondition\\\", Detail=\\\"Can't access
cart storage. System.ApplicationException: Wasn't able to connect to
redis\\n at cartservice.cartstore.RedisCartStore.EnsureRedisConnected()
in /usr/src/app/src/cartstore/RedisCartStore.cs:line 79\\n at
cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in
/usr/src/app/src/cartstore/RedisCartStore.cs:line 157\\\")\\n at
cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in
/usr/src/app/src/cartstore/RedisCartStore.cs:line 166\\n at
cartservice.services.CartService.EmptyCart(EmptyCartRequest request,
ServerCallContext context) in
/usr/src/app/src/services/CartService.cs:line 65\",\r\n
\"grouping_key\": \"6082c58afd7fa6838ebc27eef6e04829\"\r\n },\r\n
\"message\": \"Exception (plain, .NET)\",\r\n \"url\": {\r\n \"path\":
\"/oteldemo.CartService/EmptyCart\",\r\n \"original\":
\"http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart\",\r\n
\"scheme\": \"http\",\r\n \"port\": 8080,\r\n \"domain\":
\"otel-demo-green-3-cartservice\",\r\n \"full\":
\"http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart\"\r\n
},\r\n \"labels\": {\r\n \"k8s_pod_ip\": \"192.168.62.174\",\r\n
\"k8s_pod_start_time\": \"2024-12-17T02:55:54Z\",\r\n
\"service_namespace\": \"opentelemetry-demo\",\r\n
\"k8s_deployment_name\": \"otel-demo-green-3-cartservice\"\r\n },\r\n
\"observer\": {\r\n \"hostname\": \"450fbf7ab701\",\r\n \"type\":
\"apm-server\",\r\n \"version\": \"8.13.2\"\r\n },\r\n \"trace\": {\r\n
\"id\": \"699f6d1039fc873191a1c983790db3a2\"\r\n },\r\n \"@timestamp\":
\"2024-12-30T10:30:00.000Z\",\r\n \"data_stream\": {\r\n \"namespace\":
\"default\",\r\n \"type\": \"logs\",\r\n \"dataset\": \"apm.error\"\r\n
},\r\n \"service\": {\r\n \"node\": {\r\n \"name\":
\"80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359\"\r\n
},\r\n \"framework\": {\r\n \"name\": \"Microsoft.AspNetCore\"\r\n
},\r\n \"name\": \"synth-node-0\",\r\n \"language\": {\r\n \"name\":
\"dotnet\"\r\n }\r\n },\r\n \"host\": {\r\n \"hostname\":
\"ip-192-168-55-136.ec2.internal\",\r\n \"name\":
\"ip-192-168-55-136.ec2.internal\"\r\n },\r\n \"client\": {\r\n
\"port\": 35136,\r\n \"ip\": \"::ffff:192.168.50.132\"\r\n },\r\n
\"http\": {\r\n \"request\": {\r\n \"method\": \"POST\"\r\n },\r\n
\"response\": {\r\n \"status_code\": 200\r\n },\r\n \"version\":
\"2\"\r\n },\r\n \"event\": {\r\n \"agent_id_status\": \"missing\",\r\n
\"ingested\": \"2024-12-18T18:04:22Z\"\r\n },\r\n \"transaction\": {\r\n
\"id\": \"f39639f56a540e0e\",\r\n \"type\": \"request\",\r\n
\"sampled\": true\r\n },\r\n \"user_agent\": {\r\n \"original\":
\"grpc-go/1.59.0\",\r\n \"name\": \"Other\",\r\n \"device\": {\r\n
\"name\": \"Other\"\r\n }\r\n },\r\n \"span\": {\r\n \"id\":
\"f39639f56a540e0e\"\r\n },\r\n \"timestamp\": {\r\n \"us\":
1704892605838000\r\n }\r\n
}\r\n```\r\n</details>","sha":"12608c3659a2aaea702c1d341bd0fab30f968056"}},{"url":"https://github.com/elastic/kibana/pull/205254","number":205254,"branch":"8.17","state":"OPEN"},{"url":"https://github.com/elastic/kibana/pull/205255","number":205255,"branch":"8.x","state":"OPEN"}]}]
BACKPORT-->
stratoula pushed a commit to stratoula/kibana that referenced this pull request Jan 2, 2025
fixes [elastic#205240](elastic#205240)

## Summary

Fixes the issue where `error.culprit` is always missing.

<img width="800" alt="image"
src="https://github.com/user-attachments/assets/9dad6797-dd40-4206-b508-daf8d69d653d"
/>

<img width="800" alt="image"
src="https://github.com/user-attachments/assets/6a0ac25a-bc4b-4a26-b126-86eecbe7e769"
/>



### How to test 

- Run `simple_trace` synthtrace scenario
- Run the request below
<details>
<summary>POST logs-apm.error-default/_doc</summary>

```
POST logs-apm.error-default/_doc
{

    "container": {
      "id": "80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359"
    },
    "kubernetes": {
      "node": {
        "name": "ip-192-168-55-136.ec2.internal"
      },
      "pod": {
        "uid": "9bbfe3dc-dace-4638-b8f5-aa6ccf435cbe",
        "name": "otel-demo-green-3-cartservice-5d5ccc86dd-gcxtr"
      },
      "namespace": "otel-demo-green-3"
    },
    "parent": {
      "id": "f39639f56a540e0e"
    },
    "agent": {
      "name": "opentelemetry/dotnet",
      "version": "1.6.0"
    },
    "source": {
      "port": 35136,
      "ip": "::ffff:192.168.50.132"
    },
    "error": {
      "exception": [
        {
          "handled": true,
          "message": "Exception (plain, .NET) without culprit",
          "type": "Grpc.Core.RpcException"
        }
      ],
      "culprit": "culprit msg", // remove to test the N/A scenario
      "id": "60674e9b23c8b06d83889e63ffe82c1c", //change the id on every run
      "stack_trace": "Grpc.Core.RpcException: Status(StatusCode=\"FailedPrecondition\", Detail=\"Can't access cart storage. System.ApplicationException: Wasn't able to connect to redis\n   at cartservice.cartstore.RedisCartStore.EnsureRedisConnected() in /usr/src/app/src/cartstore/RedisCartStore.cs:line 79\n   at cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in /usr/src/app/src/cartstore/RedisCartStore.cs:line 157\")\n   at cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in /usr/src/app/src/cartstore/RedisCartStore.cs:line 166\n   at cartservice.services.CartService.EmptyCart(EmptyCartRequest request, ServerCallContext context) in /usr/src/app/src/services/CartService.cs:line 65",
      "grouping_key": "6082c58afd7fa6838ebc27eef6e04829"
    },
    "message": "Exception (plain, .NET)",
    "url": {
      "path": "/oteldemo.CartService/EmptyCart",
      "original": "http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart",
      "scheme": "http",
      "port": 8080,
      "domain": "otel-demo-green-3-cartservice",
      "full": "http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart"
    },
    "labels": {
      "k8s_pod_ip": "192.168.62.174",
      "k8s_pod_start_time": "2024-12-17T02:55:54Z",
      "service_namespace": "opentelemetry-demo",
      "k8s_deployment_name": "otel-demo-green-3-cartservice"
    },
    "observer": {
      "hostname": "450fbf7ab701",
      "type": "apm-server",
      "version": "8.13.2"
    },
    "trace": {
      "id": "699f6d1039fc873191a1c983790db3a2"
    },
    "@timestamp": "2024-12-30T10:30:00.000Z",
    "data_stream": {
      "namespace": "default",
      "type": "logs",
      "dataset": "apm.error"
    },
    "service": {
      "node": {
        "name": "80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359"
      },
      "framework": {
        "name": "Microsoft.AspNetCore"
      },
       "name": "synth-node-0",
      "language": {
        "name": "dotnet"
      }
    },
    "host": {
      "hostname": "ip-192-168-55-136.ec2.internal",
      "name": "ip-192-168-55-136.ec2.internal"
    },
    "client": {
      "port": 35136,
      "ip": "::ffff:192.168.50.132"
    },
    "http": {
      "request": {
        "method": "POST"
      },
      "response": {
        "status_code": 200
      },
      "version": "2"
    },
    "event": {
      "agent_id_status": "missing",
      "ingested": "2024-12-18T18:04:22Z"
    },
    "transaction": {
      "id": "f39639f56a540e0e",
      "type": "request",
      "sampled": true
    },
    "user_agent": {
      "original": "grpc-go/1.59.0",
      "name": "Other",
      "device": {
        "name": "Other"
      }
    },
    "span": {
      "id": "f39639f56a540e0e"
    },
    "timestamp": {
      "us": 1704892605838000
    }
  }
```
</details>
benakansara pushed a commit to benakansara/kibana that referenced this pull request Jan 2, 2025
fixes [elastic#205240](elastic#205240)

## Summary

Fixes the issue where `error.culprit` is always missing.

<img width="800" alt="image"
src="https://github.com/user-attachments/assets/9dad6797-dd40-4206-b508-daf8d69d653d"
/>

<img width="800" alt="image"
src="https://github.com/user-attachments/assets/6a0ac25a-bc4b-4a26-b126-86eecbe7e769"
/>



### How to test 

- Run `simple_trace` synthtrace scenario
- Run the request below
<details>
<summary>POST logs-apm.error-default/_doc</summary>

```
POST logs-apm.error-default/_doc
{

    "container": {
      "id": "80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359"
    },
    "kubernetes": {
      "node": {
        "name": "ip-192-168-55-136.ec2.internal"
      },
      "pod": {
        "uid": "9bbfe3dc-dace-4638-b8f5-aa6ccf435cbe",
        "name": "otel-demo-green-3-cartservice-5d5ccc86dd-gcxtr"
      },
      "namespace": "otel-demo-green-3"
    },
    "parent": {
      "id": "f39639f56a540e0e"
    },
    "agent": {
      "name": "opentelemetry/dotnet",
      "version": "1.6.0"
    },
    "source": {
      "port": 35136,
      "ip": "::ffff:192.168.50.132"
    },
    "error": {
      "exception": [
        {
          "handled": true,
          "message": "Exception (plain, .NET) without culprit",
          "type": "Grpc.Core.RpcException"
        }
      ],
      "culprit": "culprit msg", // remove to test the N/A scenario
      "id": "60674e9b23c8b06d83889e63ffe82c1c", //change the id on every run
      "stack_trace": "Grpc.Core.RpcException: Status(StatusCode=\"FailedPrecondition\", Detail=\"Can't access cart storage. System.ApplicationException: Wasn't able to connect to redis\n   at cartservice.cartstore.RedisCartStore.EnsureRedisConnected() in /usr/src/app/src/cartstore/RedisCartStore.cs:line 79\n   at cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in /usr/src/app/src/cartstore/RedisCartStore.cs:line 157\")\n   at cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in /usr/src/app/src/cartstore/RedisCartStore.cs:line 166\n   at cartservice.services.CartService.EmptyCart(EmptyCartRequest request, ServerCallContext context) in /usr/src/app/src/services/CartService.cs:line 65",
      "grouping_key": "6082c58afd7fa6838ebc27eef6e04829"
    },
    "message": "Exception (plain, .NET)",
    "url": {
      "path": "/oteldemo.CartService/EmptyCart",
      "original": "http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart",
      "scheme": "http",
      "port": 8080,
      "domain": "otel-demo-green-3-cartservice",
      "full": "http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart"
    },
    "labels": {
      "k8s_pod_ip": "192.168.62.174",
      "k8s_pod_start_time": "2024-12-17T02:55:54Z",
      "service_namespace": "opentelemetry-demo",
      "k8s_deployment_name": "otel-demo-green-3-cartservice"
    },
    "observer": {
      "hostname": "450fbf7ab701",
      "type": "apm-server",
      "version": "8.13.2"
    },
    "trace": {
      "id": "699f6d1039fc873191a1c983790db3a2"
    },
    "@timestamp": "2024-12-30T10:30:00.000Z",
    "data_stream": {
      "namespace": "default",
      "type": "logs",
      "dataset": "apm.error"
    },
    "service": {
      "node": {
        "name": "80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359"
      },
      "framework": {
        "name": "Microsoft.AspNetCore"
      },
       "name": "synth-node-0",
      "language": {
        "name": "dotnet"
      }
    },
    "host": {
      "hostname": "ip-192-168-55-136.ec2.internal",
      "name": "ip-192-168-55-136.ec2.internal"
    },
    "client": {
      "port": 35136,
      "ip": "::ffff:192.168.50.132"
    },
    "http": {
      "request": {
        "method": "POST"
      },
      "response": {
        "status_code": 200
      },
      "version": "2"
    },
    "event": {
      "agent_id_status": "missing",
      "ingested": "2024-12-18T18:04:22Z"
    },
    "transaction": {
      "id": "f39639f56a540e0e",
      "type": "request",
      "sampled": true
    },
    "user_agent": {
      "original": "grpc-go/1.59.0",
      "name": "Other",
      "device": {
        "name": "Other"
      }
    },
    "span": {
      "id": "f39639f56a540e0e"
    },
    "timestamp": {
      "us": 1704892605838000
    }
  }
```
</details>
cqliu1 pushed a commit to cqliu1/kibana that referenced this pull request Jan 2, 2025
fixes [elastic#205240](elastic#205240)

## Summary

Fixes the issue where `error.culprit` is always missing.

<img width="800" alt="image"
src="https://github.com/user-attachments/assets/9dad6797-dd40-4206-b508-daf8d69d653d"
/>

<img width="800" alt="image"
src="https://github.com/user-attachments/assets/6a0ac25a-bc4b-4a26-b126-86eecbe7e769"
/>



### How to test 

- Run `simple_trace` synthtrace scenario
- Run the request below
<details>
<summary>POST logs-apm.error-default/_doc</summary>

```
POST logs-apm.error-default/_doc
{

    "container": {
      "id": "80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359"
    },
    "kubernetes": {
      "node": {
        "name": "ip-192-168-55-136.ec2.internal"
      },
      "pod": {
        "uid": "9bbfe3dc-dace-4638-b8f5-aa6ccf435cbe",
        "name": "otel-demo-green-3-cartservice-5d5ccc86dd-gcxtr"
      },
      "namespace": "otel-demo-green-3"
    },
    "parent": {
      "id": "f39639f56a540e0e"
    },
    "agent": {
      "name": "opentelemetry/dotnet",
      "version": "1.6.0"
    },
    "source": {
      "port": 35136,
      "ip": "::ffff:192.168.50.132"
    },
    "error": {
      "exception": [
        {
          "handled": true,
          "message": "Exception (plain, .NET) without culprit",
          "type": "Grpc.Core.RpcException"
        }
      ],
      "culprit": "culprit msg", // remove to test the N/A scenario
      "id": "60674e9b23c8b06d83889e63ffe82c1c", //change the id on every run
      "stack_trace": "Grpc.Core.RpcException: Status(StatusCode=\"FailedPrecondition\", Detail=\"Can't access cart storage. System.ApplicationException: Wasn't able to connect to redis\n   at cartservice.cartstore.RedisCartStore.EnsureRedisConnected() in /usr/src/app/src/cartstore/RedisCartStore.cs:line 79\n   at cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in /usr/src/app/src/cartstore/RedisCartStore.cs:line 157\")\n   at cartservice.cartstore.RedisCartStore.EmptyCartAsync(String userId) in /usr/src/app/src/cartstore/RedisCartStore.cs:line 166\n   at cartservice.services.CartService.EmptyCart(EmptyCartRequest request, ServerCallContext context) in /usr/src/app/src/services/CartService.cs:line 65",
      "grouping_key": "6082c58afd7fa6838ebc27eef6e04829"
    },
    "message": "Exception (plain, .NET)",
    "url": {
      "path": "/oteldemo.CartService/EmptyCart",
      "original": "http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart",
      "scheme": "http",
      "port": 8080,
      "domain": "otel-demo-green-3-cartservice",
      "full": "http://otel-demo-green-3-cartservice:8080/oteldemo.CartService/EmptyCart"
    },
    "labels": {
      "k8s_pod_ip": "192.168.62.174",
      "k8s_pod_start_time": "2024-12-17T02:55:54Z",
      "service_namespace": "opentelemetry-demo",
      "k8s_deployment_name": "otel-demo-green-3-cartservice"
    },
    "observer": {
      "hostname": "450fbf7ab701",
      "type": "apm-server",
      "version": "8.13.2"
    },
    "trace": {
      "id": "699f6d1039fc873191a1c983790db3a2"
    },
    "@timestamp": "2024-12-30T10:30:00.000Z",
    "data_stream": {
      "namespace": "default",
      "type": "logs",
      "dataset": "apm.error"
    },
    "service": {
      "node": {
        "name": "80cc20ecab1b57865d7aca0f66ab75a0c6ec2c51f7c53da04a8540215369c359"
      },
      "framework": {
        "name": "Microsoft.AspNetCore"
      },
       "name": "synth-node-0",
      "language": {
        "name": "dotnet"
      }
    },
    "host": {
      "hostname": "ip-192-168-55-136.ec2.internal",
      "name": "ip-192-168-55-136.ec2.internal"
    },
    "client": {
      "port": 35136,
      "ip": "::ffff:192.168.50.132"
    },
    "http": {
      "request": {
        "method": "POST"
      },
      "response": {
        "status_code": 200
      },
      "version": "2"
    },
    "event": {
      "agent_id_status": "missing",
      "ingested": "2024-12-18T18:04:22Z"
    },
    "transaction": {
      "id": "f39639f56a540e0e",
      "type": "request",
      "sampled": true
    },
    "user_agent": {
      "original": "grpc-go/1.59.0",
      "name": "Other",
      "device": {
        "name": "Other"
      }
    },
    "span": {
      "id": "f39639f56a540e0e"
    },
    "timestamp": {
      "us": 1704892605838000
    }
  }
```
</details>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:prev-major Backport to (8.x, 8.17, 8.16) the previous major branch and other branches in development release_note:skip Skip the PR/issue when compiling release notes Team:obs-ux-infra_services Observability Infrastructure & Services User Experience Team v8.16.3 v8.17.1 v8.18.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[APM] Error culprit always shows N/A
4 participants