Non-spawned tasks (tracing futures)

[DzenanJupic]

That might sound stupid, and maybe there's already something for that. Would it be possible to have tasks, that are not spawned onto a runtime?

I've discovered tokio console, and I absolutely love it. The only thing that I'm missing is the ability to track arbitrary futures that are not directly spawned on the runtime. Especially when working with generic code, it's not always desirable to add a Send + 'static bound to all types.

Would it be reasonable to add something like a run, or wrap method to tokio::task::Builder, that instead of spawning the future, just adds additional tracing to it?

[DzenanJupic]

Looking at the code, it seems like the only thing that would be required would be to call tokio::util::trace::task(future) on the future. Would a PR be accepted?

[hawkw]

Currently, tokio-console only "cares about" the tracing spans emitted by Tokio for spawned tasks. However, we would like to enhance the console to allow it to also display tracing spans that are recorded by user code (see tokio-rs/console#297). I think the best way to add what you're asking for is not to expose Tokio's internal tracing helpers, but to implement a view for user-defined spans in the console. That way, you could use tracing's future.instrument() method to instrument any arbitrary future, or put the the #[instrument] attribute on your async fns, and view them all in a new "Spans" view in the console UI.

We could also associate user spans within a task span with the task span in the console, so you could see which task a chain of child futures were executing inside of.

[DzenanJupic]

Yes, I noticed that it only picks up tokio internal traces. If I'm not mistaken, this is the span it uses to grab the task information. And here is where it's added to the task.

My idea was to only add this information to a future without spawning it afterwards.

Somewhat like this (maybe with a named future instead)

// tokio/src/task/builder.rs
impl Builder {
   #[track_caller]
    pub async fn wrap<Fut>(self, future: Fut) -> Fut::Output
    where
        Fut: Future,
    {
        let id = crate::runtime::task::Id::next();
        #[cfg(all(
            tokio_unstable,
            tokio_taskdump,
            feature = "rt",
            target_os = "linux",
            any(target_arch = "aarch64", target_arch = "x86", target_arch = "x86_64")
        ))]
        let future = super::task::trace::Trace::root(future);
        #[cfg(all(tokio_unstable, feature = "tracing"))]
        let future = crate::util::trace::task(future, "task", _name, id.as_u64());
        future.await
    }
}

This way, it would be easily possible to debug all kinds of futures without the need to make modifications to tokio console or expose the tokio tracing internals.

[hawkw]

I don't think adding duplicates of the task span to arbitrary futures is as useful as enhancing the console to display arbitrary traces. This specific span is intended to be used for tracking tasks spawned on the runtime. Wrapping other futures in a task span would incorrectly indicate that they are also spawned as tasks, when they are not.

Furthermore, providing a general purpose utility for instrumenting futures with spans is not tokio::task's responsibility. tracing itself provides a much more full-featured API for constructing spans to instrument arbitrary futures. I think enhancing the console's ability to display user-defined tracing instrumentation is a better approach.

With that said, if you really want to mark your own spans as tasks, all you have to do is create tasks named runtime.spawn in your own code, and the console will consider them task spans, just like the spans emitted by Tokio. So, you can do that today without Tokio providing an API for it, if it's what you want.