[TRACKER] Telemetry and more telemetry

[daryllimyt]

Motivation

Add more telemetry to the stack. This could be anything from Sentry to our internal logging system.

Umbrellas

Business Intelligence

Having json logging here will help with collecting and wrangling this data.

Metrics

Usage statistics

Application

Frontend

• Need to decide what events should be tracked
• PostHog
  • Basic level
• Sentry

Backend

• aiodebug

• Identifiers

  • prefix all IDs with a tag
  • perf: Convert IDs into UUIDs

• Loggers

  • Evaluate loguru/structlog (outcome: selected loguru) Evaluate structlog and loguru #128
  • Use ContextVar for threadlocal loggers Move action loggers into ContextVars #127 (NOTE: this is now replaced by loguru's logger.contextualize)
  • Contextualized loggers with user/task information
    • Add contextual information for ASGI requests #133
    • Replace standard_logger with custom loguru logger feat: Upgrade logging #134
  • Structured ndjson logging feat: Upgrade logging #134
  • Do we need a logging service? Verdict: use disk logger with rotating handler + batch write to S3, then just use Athena + SQL
  • hyperDX/sentry (needs more evaluation)
  • PII redaction (may help SOC2) Add PII redacting loggers #61

• API Endpoints

  • Logging middleware to contextualize endpoint logs with user information https://docs.sentry.io/platforms/python/integrations/asgi/

• JSON Audit Trails (for enterprise; HIPPA, ISO270001)

• Better log platform (aws UI isn't great): e.g. Highlight.io

• Sentry

  • Python integrations: see https://docs.sentry.io/platforms/python/integrations/
    • Asyncio
    • FastAPI
    • ASGI
    • HTTPX
    • Loguru

Infrastructure

AWS

• Cloudwatch
• Log groups

Supabase

Clerk

• Slack webhooks
  • User sign-up

Sensitive informaiton

PII

• Sentry can help with this