Question: Does this allow us to return to Serilog for distributed tracing ?

[johnkattenhorn]

Hi @nblumhardt,

Long time no speak, my friend; it's been a while. Last couple of days, I've been bringing myself back up to speed with the advancements in Serilog (long-time fan); we had to move away for some clients who wanted specific features from AppInsights that we couldn't provide with Serilog directly.

My question is can we now do serilog ---> open telemetry ---> exporter of choice for distributed tracing, if so is there any code around showing an example of defining spans (and any provision for writing into HTTP headers to cross service boundaries) ?

We would love to come back to Serilog with all the pillars of our observability requirements (Logs, Tracing, Metrics etc)

[nblumhardt]

Hey John! Nice to hear from you! 👋

Currently, what we're working on is integration between Serilog and any OTLP Logs endpoint - broader metrics or traces integration isn't in scope right now, but it's an interesting possibility to consider. We've been thinking about how IDiagnosticContext (in the Serilog.Extensions.Hosting API) might become a core Serilog API, and there's a strong synergy with traces there 🤔 ....

I think step one is getting a strong, minimal-dependency implementation of Logs to 1.0.0 status, so I'm hesitant to look too far beyond that, but let's leave this ticket open and mull it over some more 🙂

[julealgon]

...broader metrics or traces integration isn't in scope right now, but it's an interesting possibility to consider.

Nic what exactly do you mean by this? Are you thinking about custom Serilog-defined solutions for tracing and for metrics? If so... may I humbly ask you to reconsider? These already have very well defined abstractions in .NET and introducing custom alternatives would only divide the community.

I've shared this sentiment before, but instead I'd really like to see you joining forces in the existing common abstractions instead of creating separate paths. The whole IDiagnosticContext thing is a prime example of that. I won't repeat all the points again but will link here for visibility:

• Leverage 'DiagnosticSources' for collecting data instead of custom 'DiagnosticContext' serilog-extensions-hosting#34

[nblumhardt]

Hi Juliano!

Sometimes interesting and unexpected things come out of an open discussion; I don't think any of us is keen to sign up for needlessly duplicating anything that works and is in wide use, but Serilog inhabits an ecosystem full of new ideas, techniques, libraries, and products. The conversation above is merely suggesting that there is new terrain to consider.

You're always welcome, now and in the future, to share your opinion on these things, too, so thanks for chiming in all the same 👍

A bit of an aside, one thing that might not be clear about Serilog is that it's a project intended to have a multi-decade lifespan. Collaboration and integration are good for this, but being self-contained is, too.

I learnt a lot about this through the lens of Autofac; many (many) years on, it's still a widely used, valued, and enjoyed part of .NET, despite at various times people advising the project to "join forces" with Castle Windsor, Unity, MEF, and I'm sure (though it was after my time) MEDI. In the long run, having some .NET projects with a measure of independence, a narrow vision, and long-term stability is a good thing for everyone.

If Serilog had coupled itself to the half-dozen other efforts that people suggested over the years, many now defunct, we'd have churned a lot of consumers, destroyed a lot of value in the extensive sink ecosystem, and gone from one incomplete implementation to another "lighter, faster" - and still incomplete version.

That's not to say Serilog operates in a vacuum or is driven by self-interest; in the early days of .NET Core the project worked closely with the .NET team to provide the only stable logging implementation that existed on the platform when it hit 1.0, and since then we've collaborated with the folks working on .NET, NLog and other libraries to make message templates a stable, consistent standard that's used across the whole ecosystem.

Serilog also has a long history of experimenting and looking outside to take in what's happening across the rest of the landscape; SerilogMetrics was built in 2015, for example. IDiagnosticContext was committed in 2019, when a lot of experimentation was happening throughout .NET in that area. I think that process will very likely continue - but don't assume that means Serilog's going to try to reinvent the world 😊.

Anyway, sorry about the long read, but hopefully this puts the comments above in perspective. Cheers!

[julealgon]

A bit of an aside, one thing that might not be clear about Serilog is that it's a project intended to have a multi-decade lifespan. Collaboration and integration are good for this, but being self-contained is, too.

I learnt a lot about this through the lens of Autofac; many (many) years on, it's still a widely used, valued, and enjoyed part of .NET, despite at various times people advising the project to "join forces" with Castle Windsor, Unity, MEF, and I'm sure (though it was after my time) MEDI. In the long run, having some .NET projects with a measure of independence, a narrow vision, and long-term stability is a good thing for everyone.

If Serilog had coupled itself to the half-dozen other efforts that people suggested over the years, many now defunct, we'd have churned a lot of consumers, destroyed a lot of value in the extensive sink ecosystem, and gone from one incomplete implementation to another "lighter, faster" - and still incomplete version.

You might've misunderstood me with the examples there with joining forces with other third party container libraries. I'm not suggesting that here, but instead to join forces with the standardization efforts themselves (ILogger, OpenTelemetry, etc). Hopefully, none of those will be defunct in the foreseeable future, not without a proper replacement.

My concern is centered around the fact that Serilog already does some things that have native support in the common libraries with its own custom abstractions. For things which do not exist anywhere else, I'd rather see a push from the library authors here to make the capabilities available in the standard libraries and protocols than see a straight jump into custom implementations.

I totally understand this if we were talking 10 years ago, when there were no standards for logging (or dependency injection for that matter) in .NET, but the landscape has changed dramatically.

To me, it also doesn't really matter "who started it first". The standardization wins, regardless. If Serilog had a great solution to something 10 years ago, but that solution is somehow not present in the current standardization, then my opinion is that Serilog should do 1 of 2 possible things:

1. Convince the standardization people that the thing that is missing should be part of the standard
2. Adapt Serilog to the new standard

Serilog also has a long history of experimenting and looking outside to take in what's happening across the rest of the landscape; SerilogMetrics was built in 2015, for example.

This is all well and good (and it's the first time I hear about that effort at all), but again... it does not matter "who did it first", IMHO. As soon as the instruments were standardized in .NET in the Diagnostics namespace, and the standards for metrics representation and collection were standardized with OTEL, any other implementation that uses a different abstraction really don't matter much anymore: they are automatically nonstandard, custom and, in my opinion again, should be avoided for the most part.

...but don't assume that means Serilog's going to try to reinvent the world 😊.

I hope it doesn't try to, but I do see some of it. All I'll say on this is that, the more the library attempts to vendor-lock me in, the less tempted I'll get to keep using it. Sometimes it feels to me that decisions are made in regards to Serilog with the explicit intent to force vendor lock-in.

Even the examples for the library are always focused on the Serilog's custom logger interface, instead of focusing on the .NET's standard ILogger abstraction. This just drives less senior devs into locking themselves into the library while at the same time avoiding a common, well-known abstraction that should always be encouraged vs custom implementations. Again, it does not matter that "Serilog's abstraction was created X years earlier...". As soon as the standard was introduced, the standard should be encouraged by everyone.

[nblumhardt]

As soon as the standard was introduced, the standard should be encouraged by everyone.

My post above sets out why I think this position is false, so it's difficult for me to reply without sending us around in a loop.

It's also ironic that this thread is part of a project in which a bunch of Serilog contributors are working to support yet another new standard, and that the main point of "standardization" you're discussing - .NET's ILogger<T> - has been supported by Serilog longer than any other implementer.

Happy agreeing to disagree, at this point.

[julealgon]

As soon as the standard was introduced, the standard should be encouraged by everyone.

My post above sets out why I think this position is false, so it's difficult for me to reply without sending us around in a loop.

Fair enough.

It's also ironic that this thread is part of a project in which a bunch of Serilog contributors are working to support yet another new standard...

I'm not sure why you are mentioning this considering you know I agree this is an awesome effort, that I was the one who pushed for it initially, and that I'm eagerly waiting for a RC/final release to start using it myself.

The fact that Serilog does follow some standards doesn't mean we can't talk about the instances where it doesn't, right?

... and that the main point of "standardization" you're discussing - .NET's ILogger<T> - has been supported by Serilog longer than any other implementer.

To be clear: I never said Serilog didn't support ILogger. My contention was with the fact that it doesn't encourage it's use as the default, and instead pushes the custom Serilog abstraction/implementation on people, creating vendor lock-in. To me, this is sending people to the pit of failure, as they end up introducing a vendor-specific abstraction throughout their codebase, sometimes unknowingly (see the "junior devs" argument above). You obviously know this, and yet you still push it.

I guess it comes back to your initial disagreement above.

Happy agreeing to disagree, at this point.

Also fair. As always, thanks for listening.

[nblumhardt]

Happy 2024!

We recently ported our internal use of SerilogTimings to a custom integration between Serilog and System.Diagnostics.Activity.

It came out rather well - very minimal code to get a sprinkling of timed operations and distributed traces from our existing codebase, all going through the same Serilog enrichers and sinks.

Currently working on another version of it in https://github.com/serilog-tracing/serilog-tracing - there are still some unanswered questions and rather large TODOs, but please drop by and check it out if you're interested in this space :-)