feat(cli) add telemetry scaffolding #5321

bjoerge · 2023-12-04T19:51:25Z

Description

Adds telemetry scaffolding to the cli package. The actual instrumentation is added in #5320, CLI commands for opting in and out of telemetry is added in #5322.

A tagged release of this and the other relevant PRs for telemetry can be tested using:

npm i -g sanity@telemetry

…and then running against staging (for now):

SANITY_INTERNAL_ENV=staging sanity <command> [<args>]

Other relevant env vars:

SANITY_TELEMETRY_INSPECT=1: append all logged events to telemetry-events.ndjson in current workdir (note: the output file will not be cleared between cli runs)
DEBUG=sanity:cli:telemetry: log telemetry related debug information as the CLI runs.

E.g. to test the create project flow, with debug info and logging events to file, you can use:

SANITY_INTERNAL_ENV=staging DEBUG=sanity:cli:telemetry SANITY_TELEMETRY_INSPECT=1 npm create sanity@telemetry

What to review

Does the code changes make sense?
Any potential for accidental breakage?

Notes for release

TBD

vercel · 2023-12-04T19:51:33Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
performance-studio	✅ Ready (Inspect)	Visit Preview		Dec 18, 2023 1:54pm
test-studio	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Dec 18, 2023 1:54pm

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
studio-workshop	⬜️ Ignored (Inspect)	Visit Preview		Dec 18, 2023 1:54pm

socket-security · 2023-12-04T19:52:01Z

New dependencies detected. Learn more about Socket for GitHub ↗︎

Packages	Version	New capabilities	Transitives	Size	Publisher
@sanity/telemetry	0.7.0	None	`+4`	537 kB	bjoerge

github-actions · 2023-12-04T19:55:48Z

No changes to documentation

github-actions · 2023-12-04T19:58:02Z

Component Testing Report Updated Dec 18, 2023 1:55 PM (UTC)

File	Status	Duration	Passed
comments/CommentInput.spec.tsx	✅ Passed (Inspect)	14s	15
formBuilder/ArrayInput.spec.tsx	✅ Passed (Inspect)	2s	3
formBuilder/inputs/PortableText/Annotations.spec.tsx	✅ Passed (Inspect)	8s	3
formBuilder/inputs/PortableText/Decorators.spec.tsx	✅ Passed (Inspect)	6s	6
formBuilder/inputs/PortableText/FocusTracking.spec.tsx	✅ Passed (Inspect)	16s	15
formBuilder/inputs/PortableText/Input.spec.tsx	✅ Passed (Inspect)	8s	9
formBuilder/inputs/PortableText/ObjectBlock.spec.tsx	✅ Passed (Inspect)	39s	18
formBuilder/inputs/PortableText/Styles.spec.tsx	✅ Passed (Inspect)	6s	6
formBuilder/inputs/PortableText/Toolbar.spec.tsx	✅ Passed (Inspect)	3s	3

bjoerge · 2023-12-11T12:32:09Z

Current dependencies on/for this PR:

next
- PR feat(cli) add telemetry scaffolding #5321 👈
  - PR feat(cli): treat telemetry consent as denied when running in CI #5353
  - PR feat(cli): instrument cli with telemetry events #5320
    - PR feat(cli): add telemetry cli commands #5322
      - PR feat(cli): add env var for logging telemetry events to file #5347

This stack of pull requests is managed by Graphite.

ricokahler

looks good! will review again once this points to prod instead of staging

ricokahler · 2023-12-15T18:00:34Z

packages/@sanity/cli/src/util/isCi.ts

@@ -0,0 +1,5 @@
+/* eslint-disable no-process-env */
+export const isCi =
+  process.env.CI || // Travis CI, CircleCI, Gitlab CI, Appveyor, CodeShip


would it be better to wrap these each in isTrueish?

Great callout. Will fix.

Fixed (left process.env.BUILD_NUMBER out since I don't know for sure whether BUILD_NUMBER=0 is a valid status)

ricokahler · 2023-12-15T18:08:27Z

packages/@sanity/cli/src/CommandRunner.ts

@@ -81,11 +81,7 @@ export class CommandRunner {
    const output = this.handlers.outputter
    const prompt = this.handlers.prompter

-    const {cliConfig, ...commandOptions} = options
-    const apiClient = getClientWrapper(


more curious than anything: why did you move the apiClient initialization into the CommandRunner? I didn't see this new key used anywhere in the PR.

Good question. It made sense at an earlier point during development. Reverting this. Thanks!

ricokahler · 2023-12-15T18:12:31Z

packages/@sanity/cli/src/cli.ts

@@ -76,6 +87,11 @@ export async function runCli(cliRoot: string, {cliVersion}: {cliVersion: string}
    args.groupOrCommand = 'help'
  }

+  if (args.groupOrCommand === 'logout') {
+    // flush telemetry events before logging out
+    await flushTelemetry()


why is this here instead of in the logoutCommand code?

trying to keep the interface exposed to subcommands minimal, so wanted to avoid passing the flush method to subcommands at this point. We might change this later if we see that it's something we want subcommands to be able to do, but for now it made sense to special case this here.

packages/@sanity/cli/src/util/createTelemetryStore.ts

bjoerge · 2023-12-18T13:22:42Z

packages/@sanity/cli/src/cli.ts

@@ -76,6 +87,11 @@ export async function runCli(cliRoot: string, {cliVersion}: {cliVersion: string}
    args.groupOrCommand = 'help'
  }

+  if (args.groupOrCommand === 'logout') {
+    // flush telemetry events before logging out
+    await flushTelemetry()


trying to keep the interface exposed to subcommands minimal, so wanted to avoid passing the flush method to subcommands at this point. We might change this later if we see that it's something we want subcommands to be able to do, but for now it made sense to special case this here.

bjoerge · 2023-12-18T13:23:23Z

packages/@sanity/cli/src/util/isCi.ts

@@ -0,0 +1,5 @@
+/* eslint-disable no-process-env */
+export const isCi =
+  process.env.CI || // Travis CI, CircleCI, Gitlab CI, Appveyor, CodeShip


Great callout. Will fix.

bjoerge · 2023-12-18T13:32:14Z

packages/@sanity/cli/src/CommandRunner.ts

@@ -81,11 +81,7 @@ export class CommandRunner {
    const output = this.handlers.outputter
    const prompt = this.handlers.prompter

-    const {cliConfig, ...commandOptions} = options
-    const apiClient = getClientWrapper(


Good question. It made sense at an earlier point during development. Reverting this. Thanks!

* fix(deps): add telemetry package as cli dependency * feat(cli): add telemetry scaffolding for CLI * fix(cli): flush telemetry store before logging out * refactor(cli): abstract continuous integration check * feat(cli): treat telemetry consent as denied when running in CI * fix(cli): parse trueish env vars when detecting ci environment * fixup! feat(cli): add telemetry scaffolding for CLI --------- Co-authored-by: Ash <[email protected]>

vercel bot deployed to Preview – performance-studio December 4, 2023 19:52 View deployment

vercel bot deployed to Preview – test-studio December 4, 2023 19:52 View deployment

vercel bot deployed to Preview – performance-studio December 7, 2023 10:43 View deployment

vercel bot deployed to Preview – test-studio December 7, 2023 10:43 View deployment

bjoerge force-pushed the feat/cli-telemetry-scaffolding branch from c1aeedb to 4b68c44 Compare December 7, 2023 10:56

vercel bot deployed to Preview – performance-studio December 7, 2023 10:56 View deployment

vercel bot deployed to Preview – test-studio December 7, 2023 10:56 View deployment

bjoerge force-pushed the feat/cli-telemetry-scaffolding branch from 4b68c44 to 675781c Compare December 11, 2023 12:31

This was referenced Dec 11, 2023

feat(cli): instrument cli with telemetry events #5320

Merged

feat(cli): add telemetry cli commands #5322

Merged

vercel bot deployed to Preview – performance-studio December 11, 2023 12:36 View deployment

vercel bot deployed to Preview – test-studio December 11, 2023 12:37 View deployment

bjoerge force-pushed the feat/cli-telemetry-scaffolding branch from 675781c to ea8db9e Compare December 11, 2023 14:24

vercel bot deployed to Preview – performance-studio December 11, 2023 14:28 View deployment

vercel bot deployed to Preview – test-studio December 11, 2023 14:29 View deployment

bjoerge force-pushed the feat/cli-telemetry-scaffolding branch from ea8db9e to a919840 Compare December 11, 2023 14:30

vercel bot deployed to Preview – performance-studio December 11, 2023 14:34 View deployment

vercel bot deployed to Preview – test-studio December 11, 2023 14:34 View deployment

bjoerge mentioned this pull request Dec 11, 2023

feat(cli): add env var for logging telemetry events to file #5347

Merged

juice49 force-pushed the feat/cli-telemetry-scaffolding branch from a919840 to 36311e8 Compare December 12, 2023 12:42

juice49 mentioned this pull request Dec 12, 2023

feat(cli): treat telemetry consent as denied when running in CI #5353

Closed

vercel bot deployed to Preview – performance-studio December 12, 2023 12:47 View deployment

vercel bot deployed to Preview – test-studio December 12, 2023 12:47 View deployment

bjoerge force-pushed the feat/cli-telemetry-scaffolding branch from 36311e8 to 6c225a7 Compare December 14, 2023 10:41

vercel bot deployed to Preview – performance-studio December 14, 2023 10:45 View deployment

vercel bot deployed to Preview – performance-studio December 15, 2023 10:43 View deployment

vercel bot deployed to Preview – performance-studio December 15, 2023 11:18 View deployment

vercel bot deployed to Preview – test-studio December 15, 2023 11:19 View deployment

bjoerge marked this pull request as ready for review December 15, 2023 12:57

bjoerge requested a review from a team as a code owner December 15, 2023 12:57

bjoerge requested review from ricokahler and removed request for a team December 15, 2023 12:57

ricokahler reviewed Dec 15, 2023

View reviewed changes

fix(deps): add telemetry package as cli dependency

7c31555

bjoerge force-pushed the feat/cli-telemetry-scaffolding branch from dfab462 to d86ed77 Compare December 18, 2023 13:18

vercel bot deployed to Preview – performance-studio December 18, 2023 13:22 View deployment

vercel bot deployed to Preview – test-studio December 18, 2023 13:23 View deployment

bjoerge commented Dec 18, 2023

View reviewed changes

bjoerge and others added 4 commits December 18, 2023 14:36

feat(cli): add telemetry scaffolding for CLI

d6a29d2

fix(cli): flush telemetry store before logging out

5c1775d

refactor(cli): abstract continuous integration check

520f583

feat(cli): treat telemetry consent as denied when running in CI

ec652fe

bjoerge force-pushed the feat/cli-telemetry-scaffolding branch from d86ed77 to ec652fe Compare December 18, 2023 13:41

bjoerge added 2 commits December 18, 2023 14:43

fix(cli): parse trueish env vars when detecting ci environment

0727413

fixup! feat(cli): add telemetry scaffolding for CLI

323cf72

bjoerge requested a review from ricokahler December 18, 2023 13:48

vercel bot deployed to Preview – performance-studio December 18, 2023 13:52 View deployment

vercel bot deployed to Preview – test-studio December 18, 2023 13:54 View deployment

ricokahler approved these changes Dec 18, 2023

View reviewed changes

bjoerge added this pull request to the merge queue Dec 18, 2023

Merged via the queue into next with commit 0865664 Dec 18, 2023
40 checks passed

bjoerge deleted the feat/cli-telemetry-scaffolding branch December 18, 2023 18:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(cli) add telemetry scaffolding #5321

feat(cli) add telemetry scaffolding #5321

bjoerge commented Dec 4, 2023 •

edited

Loading

vercel bot commented Dec 4, 2023 •

edited

Loading

socket-security bot commented Dec 4, 2023 •

edited

Loading

github-actions bot commented Dec 4, 2023 •

edited

Loading

github-actions bot commented Dec 4, 2023 •

edited

Loading

bjoerge commented Dec 11, 2023 •

edited

Loading

ricokahler left a comment •

edited

Loading

ricokahler Dec 15, 2023

bjoerge Dec 18, 2023

bjoerge Dec 18, 2023

ricokahler Dec 15, 2023

bjoerge Dec 18, 2023

ricokahler Dec 15, 2023

bjoerge Dec 18, 2023 •

edited

Loading

bjoerge Dec 18, 2023 •

edited

Loading

bjoerge Dec 18, 2023

bjoerge Dec 18, 2023

feat(cli) add telemetry scaffolding #5321

feat(cli) add telemetry scaffolding #5321

Conversation

bjoerge commented Dec 4, 2023 • edited Loading

Description

What to review

Notes for release

vercel bot commented Dec 4, 2023 • edited Loading

socket-security bot commented Dec 4, 2023 • edited Loading

github-actions bot commented Dec 4, 2023 • edited Loading

github-actions bot commented Dec 4, 2023 • edited Loading

bjoerge commented Dec 11, 2023 • edited Loading

ricokahler left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bjoerge Dec 18, 2023 • edited Loading

Choose a reason for hiding this comment

bjoerge Dec 18, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bjoerge commented Dec 4, 2023 •

edited

Loading

vercel bot commented Dec 4, 2023 •

edited

Loading

socket-security bot commented Dec 4, 2023 •

edited

Loading

github-actions bot commented Dec 4, 2023 •

edited

Loading

github-actions bot commented Dec 4, 2023 •

edited

Loading

bjoerge commented Dec 11, 2023 •

edited

Loading

ricokahler left a comment •

edited

Loading

bjoerge Dec 18, 2023 •

edited

Loading

bjoerge Dec 18, 2023 •

edited

Loading