How should we capture telemetry?

[mefellows]

With the introduction of Google Analytics 4, our current telemetry system is no longer capturing usage metrics and we are missing important information to help us make better product decisions.

Given there is work required to migrate to GA4, it is a timely opportunity to review our approach and tools used. This Q&A invites the community to contribute ideas toward a modern telemetry system that serves the needs of the community, including respecting the privacy of individuals and companies.

[JP-Ellis]

Not a direct answer, but it would be good to see how other open source packages capture telemetry. I'll expand on this post as I find more packages which use telemetry.

Azure CLI

Data Collected

I was unable to find exactly what it collects.

From inspecting the source code, they upload usage data to Azure Application Insight.

AWS SAM CLI

According to AWS' documentation, the CLI collected is:

Usage information – The generic commands and subcommands that customers run.

Errors and diagnostic information – The status and duration of commands that customers run, including exit codes, internal exception names, and failures when connecting to Docker.

System and environment information – The Python version, operating system (Windows, Linux, or macOS), environment in which the AWS SAM CLI runs (for example, AWS CodeBuild, an AWS IDE toolkit, or a terminal), and hash values of usage attributes.

The source code shows that they upload telemetry to:

https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics

I do not know what is behind that endpoint.

[mefellows]

Thanks! Yes, we looked at SAM and dotnet (updated: not Azure) of these when creating our own. The dotnet CLI publishes what they send here: https://learn.microsoft.com/en-us/dotnet/core/tools/telemetry