Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Base telemetry #969

Open
valentimarco opened this issue Nov 7, 2024 · 4 comments · May be fixed by #965
Open

Base telemetry #969

valentimarco opened this issue Nov 7, 2024 · 4 comments · May be fixed by #965
Labels
enhancement New feature or request

Comments

@valentimarco
Copy link
Collaborator

We need to collect same basic metrics about Cat usage.

Main idea:

  • enable telemetry by default with the option to optout
  • inform users that telemetry is active? when should this happen?

Possible main metrics:

  • country, where in the world is the Cat being used?
  • version, which versions are users on?
  • unique id of installation, how many instanced of the Cat are roaming in the world?(A GUID generated on first run can serve as a unique ID, without requiring an actual unique identifier for each installation)
  • configured LLM model, what is the most used LLM model? are users using local LLMs?

Additional metrics, in order of priority:

  • max number of concurrent sessions, is the Cat used as a personal assistant or as a shared resource accessible to multiple users?
  • admin portal usage, is the user accessing the chat through the admin portal or by other means?
  • number of plugins installed, is Cat being used as a "simple" rag or for execute actions? (this is still a very draft idea...)

Related Discussion:

  • Would it be useful to display some of these metrics within the admin portal?
@valentimarco valentimarco converted this from a draft issue Nov 7, 2024
@valentimarco valentimarco added the enhancement New feature or request label Nov 7, 2024
@valentimarco valentimarco linked a pull request Nov 7, 2024 that will close this issue
[Feature] Add telemetry #965
Draft
7 tasks
@leomanga
Copy link

leomanga commented Nov 9, 2024

I have started writing a server to receive and store telemetry data: https://github.com/leomanga/Telemetry-Server-for-cheshire
What type of database would be suitable for this purpose? Could MongoDB be okay?

@pieroit
Copy link
Member

pieroit commented Nov 9, 2024

I have started writing a server to receive and store telemetry data: https://github.com/leomanga/Telemetry-Server-for-cheshire What type of database would be suitable for this purpose? Could MongoDB be okay?

Awesome thanks for sharing @leomanga!
Yep I guess mongo will do, as the data structure may change ;)

@leomanga
Copy link

leomanga commented Nov 9, 2024 •
edited
Loading

Awesome thanks for sharing @leomanga! Yep I guess mongo will do, as the data structure may change ;)

I've just implemented mongoDB in my repo. Are there any improvements I can make to the code?

@valentimarco
Copy link
Collaborator Author

@leomanga you can improve by:

  1. Adding a docker compose
  2. have a .env instead of variable.py for common variables
  3. Idk if MongoClient support connection pool, but the MongoDB class must be declared one time as global or defined as Singleton class

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
Cat Project Kanban
Status: 📋 Backlog
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Feature] Add telemetry valentimarco/core
3 participants
@pieroit @valentimarco @leomanga