Group by Exception Message and Exception Type

[DeanGoodfynd]

Description

Bugsnag default grouping uses the Exception class to determine how to group error reports. However, many libraries written in Microsoft DotNet use a common exception Type and distinguish the particular cause of the error by the exception Message. This can cause BugSnag to lump together bugs that have very different causes.

Describe the solution you'd like
Ideally, I'd like to designate certain exception classes for which a combination of Exception Type and Exception Message will be used as the grouping discriminator (hash?).

Note that we probably don't want to use the Exception Message but rather the Exception Message Template. Often the Exception Message contains specifics about the error thrown, which may mean that the error messages won't get grouped at all. But I suspect this situation will improve as more libraries are updated to use structured logging.
https://docs.microsoft.com/en-us/aspnet/core/fundamentals/logging/?view=aspnetcore-6.0

For example, newer versions of Visual Studio encourage the use of structured message logging templates as part of code quality standards: https://docs.microsoft.com/en-us/dotnet/fundamentals/code-analysis/quality-rules/ca2254

Describe alternatives you've considered
Maybe we don't need to specify which C# classes need to be unpacked this way? If we can determine that MessageTemplate != Message, we know that structured logging is in effect, and we can just group bugs according to the structured logging rules.

Additional context
One key reason for requesting this feature is that we make use of many third-party libraries that we've brought in via NuGet packages. We don't have control over how these libraries throw errors. Sadly many of these providers throw a single exception type and use message to distinguish what happened.

One key example from the ecommerce domain: Stripe throws nearly everything as a StripeException, but with vastly different exception messages.

[luke-belton]

Hi @DeanGoodfynd - thanks for raising this! We do already have an item on our backlog to look at grouping by error message, similar to the functionality we already offer for error context - I've noted your use case to our product team who will consider that in the future. I don't have an ETA for this work, however in the meantime I do have a couple of suggestions that you could use as a workaround.

Firstly you could consider updating the error context to the error message that you want to use for grouping, e.g. something like:

bugsnag.BeforeNotify(report => {
  report.Event.Context = report.OriginalException.Message;
});

Once you've set the context in this way, you can choose error classes in the Bugsnag dashboard that you wish to group by context on, in the usual way.

Your second option would be to consider using a custom grouping hash. This would probably look something like:

bugsnag.BeforeNotify(report => {
  if (report.OriginalException is StripeException) {
    report.Event.GroupingHash = report.OriginalException.Message;
  }
});

Hopefully you can adapt the above to work for the structured logging use case too!

[DeanGoodfynd]

Thanks, Luke. I'll investigate some of the work-arounds you mentioned. It would be great to have this included as part of the product, though! One reason why this is important to us is that we link our Bugsnag errors to our issue tracker, JIRA. When we get several issues lumped together in one group, it often means we need to link multiple JIRA tickets to the same Bugsnag report. Sometimes this can cause issues to close and reopen in our issue tracker unexpectedly as a result. Better grouping will definitely help here! Dean

…

On Mon, Aug 8, 2022 at 4:24 AM Luke Belton ***@***.***> wrote: Hi @DeanGoodfynd <https://github.com/DeanGoodfynd> - thanks for raising this! We do already have an item on our backlog to look at grouping by error message, similar to the functionality we already offer for error context - I've noted your use case to our product team who will consider that in the future. I don't have an ETA for this work, however in the meantime I do have a couple of suggestions that you could use as a workaround. Firstly you could consider updating the error context to the error message that you want to use for grouping, e.g. something like: bugsnag.BeforeNotify(report => { report.Event.Context = report.OriginalException.Message; }); Once you've set the context in this way, you can choose error classes in the Bugsnag dashboard that you wish to group by context <https://docs.bugsnag.com/product/error-grouping/#error-context> on, in the usual way. Your second option would be to consider using a custom grouping hash <https://docs.bugsnag.com/product/error-grouping/#custom-grouping-hash>. This would probably look something like: bugsnag.BeforeNotify(report => { if (report.OriginalException is StripeException) { report.Event.GroupingHash = report.OriginalException.Message; } }); Hopefully you can adapt the above to work for the structured logging use case too! — Reply to this email directly, view it on GitHub <#160 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AU5GJTKM75MGJVOKXSEVUNTVYC75TANCNFSM552UNK6A> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[luke-belton]

Understood - I've noted that with our product team, too. Do let us know how you get on with implementing the workarounds - feel free to write into [email protected] if you'd prefer to carry on the conversation there 👍

[gareththackeray]

Hi @DeanGoodfynd I'm only really aware of Message Templates with structured logging frameworks like Serilog, rather than with Exceptions. Could you explain how you get a Message Template from an Exception? Is this something you've added to your own Exception types perhaps?

[DeanGoodfynd]

Hi Gareth, We're using Microsoft.Extensions.Logging in a .NET 6 (.NET core) web api. Our configuration is pretty vanilla. I think you're looking for the "State" and "{OriginalFormat}" properties of the exception. I'm having trouble finding the official Microsoft docs on this, though. But I've provided an example of what one looks like below. If you could group bugs by the "{OriginalFormat}" field, it would provide much better granularity for grouping messages. At least for everyone taking advantage of structured logging. But now that Visual Studio / Visual Studio Code has started nagging folks about it, I expect uptake to be fairly quick. Here's our core setup in Program.cs: return WebHost.CreateDefaultBuilder(args) .UseKestrel(options => { options.Limits.KeepAliveTimeout = TimeSpan.FromMinutes(10); }) .ConfigureLogging(logging => { logging.ClearProviders(); logging.AddConsole(); logging.AddDebug(); }) .ConfigureAppConfiguration((hostingContext, configBuilder) => { configBuilder.BindAppSettings(hostingContext); }) .UseSetting("detailedErrors", "true") .CaptureStartupErrors(true) .UseStartup<Startup>(); In our appsettings.json file, we have this configuration. Other than opting in to the JSON formatter, it's also pretty standard. "Logging": { "LogLevel": { "Default": "Warning", "System": "Warning", "Microsoft": "Warning" }, "Console": { "FormatterName": "json" } }, When the exception gets serilaized to JSON, we see this in our logs: { "EventId": 0, "LogLevel": "Warning", "Category": "Fynd.Vendors.Services.VendorAdminService", "Message": "SetLocationAndTimeZone skipped due to invalid vendor location for: la-raza", "State": { "Message": "SetLocationAndTimeZone skipped due to invalid vendor location for: la-raza", "VendorNameId": "la-raza", "{OriginalFormat}": "SetLocationAndTimeZone skipped due to invalid vendor location for: {VendorNameId}" } } And the line of code that generates the above message is this: _log.LogWarning("SetLocationAndTimeZone skipped due to invalid vendor location for: {VendorNameId}", vendor.NameId); So in theory, it should be easy to group on the original format field, which should be a static message template. Dean

…

On Thu, Aug 18, 2022 at 7:14 AM Gareth Thackeray ***@***.***> wrote: Hi @DeanGoodfynd <https://github.com/DeanGoodfynd> I'm only really aware of Message Templates with structured logging frameworks like Serilog, rather than with Exceptions. Could you explain how you get a Message Template from an Exception? Is this something you've added to your own Exception types perhaps? — Reply to this email directly, view it on GitHub <#160 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AU5GJTPCJBXOQ7MMX3A5G2LVZYLH3ANCNFSM552UNK6A> . You are receiving this because you were mentioned.Message ID: ***@***.***>