Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

common: Add a debug mode for tracing #7850

Merged
merged 7 commits into from
Nov 23, 2024
Merged

common: Add a debug mode for tracing #7850

merged 7 commits into from
Nov 23, 2024

Commits on Nov 22, 2024

  1. common: Add some debuggig capabilities to the trace subsystem 

    After adding the DB query instrumentation we ran into a couple of
issues, with spans not being resumed correctly, and it was rather hard
to identify the problem. This adds debug statements so we can trace
the tracing (traception if you will).

Changelog-None
    @cdecker
    cdecker committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    60bf3d3 View commit details
    Browse the repository at this point in the history

  2. db: Fix a broken span context pointer

    @cdecker
    cdecker committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    a648439 View commit details
    Browse the repository at this point in the history

  3. common: Resume the startup trace after exiting io_loop 

    This was a bit harder to identify: during an `io_loop` run we suspend
the current span before handing over to `io_loop`, and later when a callback
is called we resume the span again. Depending on how we return from
the `io_loop` instance that is used to drive the startup, we either
have resumed the last span, or we don't. Since we start a span before
`io_loop` and want it to be emitted afterwards, we need to take care
of the case where we returned from a callback that did not resume, and
therefore the current context is empty.

Making `trace_span_resume` idempotent means we can just resume it
manually.

Ideally we'd push the suspend / resume logic down into `io_loop`
itself, and then we'd have just one place. Maybe suspend and resume
callbacks that can be configured in `io_loop`?
    @cdecker
    cdecker committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    a97d3e5 View commit details
    Browse the repository at this point in the history

  4. common: Remove the recursive parent resolution in traces 

    We have the space in memory set aside anyway, so let's just copy the
`trace_id` into the span itself, rather than resolving the `root` at
time of emission.
    @cdecker
    cdecker committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    25d83df View commit details
    Browse the repository at this point in the history

  5. common: Add a tree checker for trace spans 

    Trace spans form a tree, but we don't actually check that the
structure doesn't break. Breakage can for example come if we use the
same key accidentally, making a new span its own ancestor.
    @cdecker
    cdecker committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    f095f58 View commit details
    Browse the repository at this point in the history

  6. common: Fix a potential cycle in the trace structure 

    It turns out that under some circumstances we end up clearing the
pointee of `current` but not the pointer. Thus when we select the next
slot we can end up reusing the same slot, making it its own parent.

We forcefull break these cycles by enforcing that `current` should
never be returned and be set as its own parent.

Changelog-None
    @cdecker
    cdecker committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    fe376f2 View commit details
    Browse the repository at this point in the history

  7. common: Make trace debugging output configurate at compile time 

    Just added a couple of compile-time guards and sprinkled the invariant
checking in a couple of places (disabled if compile time guard is
unset).
    @cdecker
    cdecker committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    cc7a826 View commit details
    Browse the repository at this point in the history