Old issues being categorized as unhandled error

[daniloapr]

Platform

Dart

Obfuscation

Disabled

Debug Info

Disabled

Doctor

[✓] Flutter (Channel stable, 3.13.4, on macOS 15.0.1 24A348 darwin-arm64, locale en-BR)
• Flutter version 3.13.4 on channel stable at /Users/danilorego/fvm/versions/3.13.4
• Upstream repository https://github.com/flutter/flutter.git
• Framework revision 367f9ea16b (1 year, 1 month ago), 2023-09-12 23:27:53 -0500
• Engine revision 9064459a8b
• Dart version 3.1.2
• DevTools version 2.25.0

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
• Android SDK at /Users/danilorego/Library/Android/sdk
• Platform android-34, build-tools 34.0.0
• Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (build 17.0.11+0-17.0.11b1207.24-11852314)
• All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 16.0)
• Xcode at /Applications/Xcode.app/Contents/Developer
• Build 16A242d
• CocoaPods version 1.15.2

[✓] Chrome - develop for the web
• Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2024.1)
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 17.0.11+0-17.0.11b1207.24-11852314)

[✓] VS Code (version 1.94.1)
• VS Code at /Applications/Visual Studio Code.app/Contents
• Flutter extension version 3.98.0

[✓] Connected device (2 available)
• macOS (desktop) • macos • darwin-arm64 • macOS 15.0.1 24A348 darwin-arm64
• Chrome (web) • chrome • web-javascript • Google Chrome 129.0.6668.100
! Error: Browsing on the local area network for Danilo’s iPhone. Ensure the device is unlocked and attached with a cable or associated with the same local area network as this Mac.
The device must be opted into Developer Mode to connect wirelessly. (code -27)

[✓] Network resources
• All expected network resources are available.

• No issues found!

Version

8.9.0

Steps to Reproduce

• Upgrade from 7.11.0
• Run the project and force some exception that would be sent to sentry but wouldn't cause a crash

// Call this when press a button
void function() {
  throw Exception('Any error');
}

// In the build method:
ElevatedButton(
    onPressed: () {
        function();
    },
    child: Text('TAP ME'),
),

Expected Result

Show the exception as a handled issue in the dashboard as it was before.

Actual Result

The exception is categorized as unhandled error, affecting the crash free rate, even though it doesn't crash the app.

Are you willing to submit a PR?

No

[buenaflor]

Hi, this has been changed in v8. Bit of context: generally Sentry release health processes unhandled events as crashed. In Flutter this is not the case though because errors can be unhandled but not crashing. This is something that we are still exploring how Sentry can change the way we view unhandled vs crashed errors while being inclusive of all the supported platforms peculiarities such as native Android, iOS etc...

If you want to keep the old behaviour you can use set this flag to false which will report unhandled errors as level error instead of fatal: options.markAutomaticallyCollectedErrorsAsFatal = false

However, we believe reporting unhandled errors as fatal by default is the more accurate representation in our current (not ideal) model even though it doesn't actually crash.

[ueman]

Relevant: getsentry/rfcs#10

[daniloapr]

Hi @buenaflor! Thanks for the explanation. I can totally see these errors being unhandled, because in fact they are, as they stop the function's execution. I think the inconsistency is considering them a crash for the crash-free metrics in the dashboard (which we use in our mobile OKRs). My suggestions is to separate the unhandled and the crashes in the dashboard, with different metrics and an extra category for crashes that force close the application, if there isn's already.

[buenaflor]

My suggestions is to separate the unhandled and the crashes in the dashboard

Current issue as mentioned is that in the Sentry model an unhandled issue = crash, which is true for other platforms such as native Android or iOS but not Flutter.

The most ideal solution as you said would be to have a mechanism in place to separate those. This has been an ongoing issue at Sentry.

I'll discuss this internally and see if we can push this issue forward

[daniloapr]

@buenaflor Perfect, thank you!

[buenaflor]

@daniloapr however to give you a heads up, realistically this issue and solutions will take time to evolve since release health is a major part of Sentry's product and involves pretty much every SDK, not only Flutter. So the scope of this is bigger than it initially seems.

You can skim through the mentioned RFC above if you're interested in some of the discussions

[daniloapr]

Hi @buenaflor! Just following up on the markAutomaticallyCollectedErrorsAsFatal = false setting you suggested. We added it to our project, but it didn’t work as expected. Do you have any other recommendations? Also, would it be possible to separate unhandled issues from app-closing crashes on the crash-free metrics in the releases dashboard?

[buenaflor]

hey, sorry, my suggestion was not correct.

you can use options.beforeSend to change this behaviour:

// in your SentryFlutter.init
options.beforeSend = (event, hint) {
  event = event.copyWith(
    exceptions: event.exceptions
        ?.map(
          (e) => e.mechanism?.handled == false
              ? e.copyWith(
                  mechanism: e.mechanism?.copyWith(handled: true))
              : e,
        )
        .toList(),
  );
  return event;
};

This works because now every error reported from flutter will always be handled = true and only actual app crashes will be marked as unhandled by the native sdks