Track how often breadcrumbs are dropped by SDKs

[AbhiPrasad]

Description

For the upcoming logging product, we want to leverage breadcrumbs in the SDK. To get a better understanding of how breadcrumbs are created and used by the SDK, we would like to track how often breadcrumbs are dropped by the SDK.

We can do this with client outcomes in the SDK.

Why should we be doing this?

In short we want to know: "How many breadcrumbs are being dropped by Sentry". If we can get this number for at least JavaScript, Python, and a single mobile SDK, we can make much better decisions around how the logging product will proceed at Sentry.

Why now?

We're working on logs - we want to ship soon, so this is important.

RFC

No response

Slack-Channel

#proj-structured-logs

Notion Document(s)

No response

Stakeholder(s)

@getsentry/product-owners-explore-logs

Team(s)

Web Frontend

SDKs

Pre-work

Preview Give feedback

1. feat(develop): Propose new discard reason for buffer overflow sentry-docs#12395
   +0
   
2. feat(outcomes): Add buffer_overflow discard reason snuba#6792
   +0
   

SDKs

Preview Give feedback

1. feat(core): Add client outcomes for breadcrumbs buffer sentry-javascript#15082
   +0
   
2. feat(scope): Emit buffer_overflow discard reason when popping breadcrumbs sentry-python#3993
   +0
   

Post SDK Work

Preview Give feedback

1. Create dashboard to track these stats

TBD on the mobile SDK

[AbhiPrasad]

diff --git a/packages/core/src/scope.ts b/packages/core/src/scope.ts
index 995be01bb..52bd4aae5 100644
--- a/packages/core/src/scope.ts
+++ b/packages/core/src/scope.ts
@@ -481,7 +481,11 @@ export class Scope {
 
     const breadcrumbs = this._breadcrumbs;
     breadcrumbs.push(mergedBreadcrumb);
-    this._breadcrumbs = breadcrumbs.length > maxCrumbs ? breadcrumbs.slice(-maxCrumbs) : breadcrumbs;
+
+    if (breadcrumbs.length > maxCrumbs) {
+      this._client?.recordDroppedEvent('breadcrumb_dropped', 'log', 1);
+      this._breadcrumbs = breadcrumbs.slice(-maxCrumbs);
+    }
 
     this._notifyScopeListeners();

Approx implementation in the JS SDK.

We need to decide what data category the breadcrumb data belongs to, and what the outcome name will be.

[AbhiPrasad]

data categories: getsentry/relay#4455