Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to completely disable Npgsql tracing #5740

Closed
samcarton opened this issue Jun 26, 2024 · 2 comments · Fixed by #5795
Closed

Unable to completely disable Npgsql tracing #5740

samcarton opened this issue Jun 26, 2024 · 2 comments · Fixed by #5795

Comments

@samcarton
Copy link

Describe the bug

  • We have a scenario where we want to disable tracing completely for Npgsql - in this instance there is an issue with Npgsql where multiplexing and open telemetry tracing are causing runaway CPU and memory issues.
  • Simply setting DD_TRACE_Npgsql_ENABLED=false does cause datadog to ignore traces produced by Npgsql
  • However, because the datadog Tracer still listens to the Npgsql ActivitySource in the IgnoreActivityHandler, Npgsql is still producing the traces, and still causing runaway CPU and memory issues.
  • Is there any way to completely disconnect certain integrations?
  • This is hopefully a temporary issue on the Npgsql side, but in the meantime we appear to have no way to turn off the problematic tracing.
  • This is also a broader issue, in that the tracing overhead is still incurred even when you would expect an Ignored integration to disable tracing for that integration entirely, whatever your reason for doing so may be.

To Reproduce
Steps to reproduce the behavior:

  1. Disable tracing for an integration, eg DD_TRACE_Npgsql_ENABLED=false
  2. Notice traces are still being created by the code

Expected behavior

  • Ignored integrations shouldn't be tracing at all.
  • OR there should be a way to completely disable tracing for an integration, instead of the enabled-but-ignored scenario we have now.

Runtime environment (please complete the following information):

  • Instrumentation mode: Nuget
  • Tracer version: <PackageReference Include="Datadog.Trace" Version="2.45.0" />
  • CLR: .net core 8.0.5
@bouwkast
Copy link
Contributor

Hi @samcarton, at this moment the only way to disable this is to set DD_TRACE_OTEL_ENABLED to false (if it is enabled), unfortunately this would disable the OpenTelemetry support within the .NET Tracer as well.

Thanks for raising this though as we will look into improving this as this isn't ideal.

@lucaspimentel
Copy link
Member

lucaspimentel commented Aug 21, 2024 •
edited
Loading

Update: it looks like #5795 (work in progress) will help resolve this issue

@bouwkast bouwkast mentioned this issue Oct 11, 2024
Merged
bouwkast added a commit that referenced this issue Feb 21, 2025
@bouwkast
Add ability to disable ActivitySources from being listened to (#5795)
c63ed2c 
## Summary of changes

This adds the ability for `ActivitySource`s to be disabled so that they
won't be listened to by the .NET Tracer.

## Reason for change

Currently there is no way to disable an `ActivitySource` if
OpenTelemetry is enabled in the .NET Tracer without entirely disabling
OpenTelemetry support. The `IgnoreActivityHandler` is insufficient as it
will still _listen_ to the `ActivitySource` so they will create
`Activity` objects.

## Implementation details

- Adds `DD_TRACE_DISABLED_ACTIVITY_SOURCES` that accepts a
comma-separated list of `ActivitySource` names that supports the
glob-pattern.
- example: `"DD_TRACE_DISABLED_ACTIVITY_SOURCES":
"SomeSourceName,*SomeGlobPattern*"`
- Adds `DisableActivityHandler` that will determine based on the above
environment variable whether or not a given `ActivitySource` should be
listened to.

## Test coverage

- Updated the `NetActivitySdk` sample project with an ignored
`ActivitySource` and some that are disabled.

## Other details
<!-- Fixes #{issue} -->

When merged, get this merged:
DataDog/dd-go#171612

Fixes #5740 

<!-- ⚠️ Note: where possible, please obtain 2 approvals prior to
merging. Unless CODEOWNERS specifies otherwise, for external teams it is
typically best to have one review from a team member, and one review
from apm-dotnet. Trivial changes do not require 2 reviews. -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add ability to disable ActivitySources from being listened to DataDog/dd-trace-dotnet
3 participants
@samcarton @lucaspimentel @bouwkast