DEBUG-2334 remaining dynamic instrumentation code

[p-datadog]

What does this PR do?

• Adds DI Remote and integrates DI into remote config processing
• Integration test starting from remote config parsing, through instrumentation to snapshot sending to backend
• DI is now enabled by default
• DI code tracking is now enabled by default (script_compiled trace point) if DD_DYNAMIC_INSTRUMENTATION_ENABLED is set in environment

Motivation:

Initial DI implementation in Ruby

Change log entry
Dynamic instrumentation is now available in Ruby. Currently only log probes are implemented, but they can be set on both methods and lines.

Additional Notes:

Replaces #4063

How to test the change?
Additional unit and integration tests are included + system tests in DataDog/system-tests#3516

[pr-commenter]

Benchmarks

Benchmark execution time: 2024-11-22 03:56:32

Comparing candidate commit d3fe663 in PR branch di-rest-2 with baseline commit 0be95cb in branch master.

Found 1 performance improvements and 2 performance regressions! Performance is the same for 28 metrics, 2 unstable metrics.

scenario:profiler - sample timeline=false

• 🟥 throughput [-0.613op/s; -0.603op/s] or [-9.221%; -9.075%]

scenario:tracing - Propagation - Trace Context

• 🟥 throughput [-4368.546op/s; -4259.134op/s] or [-11.273%; -10.990%]

scenario:tracing - Tracing.log_correlation

• 🟩 throughput [+9731.871op/s; +10049.638op/s] or [+8.820%; +9.108%]

[marcotc]

Small nit:

Suggested change

	if RUBY_VERSION >= '2.6'
	begin
	# Activate code tracking by default because line trace points will not work
	# without it.
	Datadog::DI.activate_tracking!
	rescue => exc
	if defined?(Datadog.logger)
	Datadog.logger.warn("Failed to activate code tracking for DI: #{exc.class}: #{exc}")
	else
	# We do not have Datadog logger potentially because DI code tracker is
	# being loaded early in application boot process and the rest of datadog
	# wasn't loaded yet. Output to standard error.
	warn("Failed to activate code tracking for DI: #{exc.class}: #{exc}")
	end
	end
	return if RUBY_VERSION >= '2.6'
	# Activate code tracking by default because line trace points will not work
	# without it.
	Datadog::DI.activate_tracking!
	rescue => exc
	if defined?(Datadog.logger)
	Datadog.logger.warn("Failed to activate code tracking for DI: #{exc.class}: #{exc}")
	else
	# We do not have Datadog logger potentially because DI code tracker is
	# being loaded early in application boot process and the rest of datadog
	# wasn't loaded yet. Output to standard error.
	warn("Failed to activate code tracking for DI: #{exc.class}: #{exc}")
	end

[p-datadog]

I think you meant to reverse the sign on comparison which I did.

[marcotc]

It is unusual to see method calls done at the root level, when the file is loaded.
Can you add a short comment here explaining why it's needed at this level? I understand that it's an early activation needed for DI, and that there's a more expansive explanation of DI behavior somewhere else in code, but just leave a short note here so people don't get confused about this call site.

[p-datadog]

Added a comment and this is now conditional on DI environment variable being set.

[marcotc]

I don't think you should have this remote configuration class receiving data if DI's component was never initialize.
I'd say we can assume that this is an impossible case. It makes sense to check if DI's component is always initialized early (which I think it is).

[p-datadog]

Adjusted the comment and maybe we can inject the DI component into here at some point in the future instead of having to reference a global. I think this is the only place besides code tracker (which necessarily must reference a global, since normally it's created before DI component exists at all) that doesn't have all of its dependencies injected.

[p-datadog]

@marcotc I added a note here per your comment in the other PR, does it sound OK to you?