Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/223 billing basics #224

Merged
merged 14 commits into from
Feb 3, 2025
Merged

Feature/223 billing basics #224

merged 14 commits into from
Feb 3, 2025

Conversation

tahpot
Copy link
Member

@tahpot tahpot commented Jan 28, 2025

Closes #223

Support billing

@tahpot tahpot added the enhancement New feature or request label Jan 28, 2025
@tahpot tahpot self-assigned this Jan 28, 2025
@tahpot
Copy link
Member Author

tahpot commented Jan 28, 2025

Breaking changes:

  • Adds payer to auth request, which indicates who will pay for API requests made with an API key (either: user or app).

@tahpot tahpot requested a review from aurelticot February 2, 2025 11:47
@tahpot tahpot merged commit c4c2839 into develop Feb 3, 2025
2 checks passed
@tahpot tahpot deleted the feature/223-billing-basics branch February 3, 2025 23:52
tahpot added a commit that referenced this pull request Feb 4, 2025
* Fix CORS

* Remove auto registration of new DIDs in the waitlist + Add latest access (#166)

* Add timeout support for handlers. Fix bug with timeout on provider with incorrect usage of await. (#174)

* Fix/171 calendar event not indexed (#173)

* Feature/improve ai rag (#172)

* Support create update API (#167)

* Increment version

* Add timer for LLM search processing. Throw error if no prompt search result oject.

* Support profile prompt endpoint

* Escape HTML on the client side on the data page

* Improve handling of JSON output

* Fix personal prompt always returning JSON

* 184 support llama 32 (#185)

* Adding support for directly using AWS bedrock SDK to access Llama 3.x using inference profiles

* Support default provider and model in config

* Support LLM token limit, flagging no system prompt available for LLM, including LLM info in API responses

* Update CHANGELOG for 0.2.0 release

* Increment version

* Update CHANGELOG with missing changes

* Remove console output when local config doesn't exist

* Add a status to the provider object (#188)

* feat: add status to providers
* feat: display upcoming providers in web UI
* Skip inactive providers when fetching active connections for a user

* Add Claude Haiku 3 support

* Feature/langchain tools llm (#194)

* Add langchain endpoint that supports calendar and email query tools using claude 3.5 haiku

* Support counting query results instead of returning all results

* Createa a langchain agent

* Add tools for calendar event, chat group, email, file, following, post.

* Create the concept of a data schema. Refactor to use data schema.

* Support vector store

* Add vector store dependencies

* Cleanup dependencies, rename agent endpoint

* Remove invalid together key

* Support hotloading keyword index and /or vector db

* Rename personal prompt AI to `ai-old`

* Add timestamp and group id to schemas

* Add chat message and favourite schemas

* Prepare for filter support to vector store

* Summarize long emails. Force remove HTML from email raw text. Agent endpoint returns tools, latency and token information. Replace vector search with keyword search for performance. Ensure agent tools restrict output by a max token count parameter

* Support web searching via tools. Include tool processing info and token usage in output.

* Remove tiktoken no longer used

* Add support for fetching specific database records by ID. Give the LLM the current date/time. Add ID and sourceApplication property to LLM tools.

* Remove vector store dependency

* Support chat threads being fetched properly

* Make LLM avoid generating case insenistive pouchdb queries as they aren't supported. Fix chat threads not being included in the context properly.

* Update to latest verida protocol release

* Update personal profile endpoint to use agent

* Improve agent prompt to provide better markdown output

* Fix/200 sync issues (#203)

* Fix timeout detection always incorrect due to handler start log message
* Support handler v connection timeouts. Fix timeouts never being detected due to log timing issue.

* Update CHANGELOG in preparation for 0.3.0 release

* Add github, slack, spotify, discord as upcoming connectors

* Output provider descriptions from the config in the `/providers` endpoint

* Feature/74 oauth support (#134)

* Remove unused Telegram dependencies

* PoC in progress

* Draft KMS for Oauth

* Start of server, user, client objects with updated routes

* Add better architecture for OAuth implementation. Add basic unit test that initiaties auth.

* Basic flow working with hardcoded variables

* Progress on performing verification checks on auth request

* Implement most of request verification

* Move interfaces to separate file. Implement database functions for authorization tokens.

* Support token revocation. Improve security. All functionality has passing unit tests. Code cleanup.

* Refactor to have AuthUser object. Support /token and /tokens endpoints. Code cleanup.

* Rename oauth to auth. Remove redundant files.

* Extract auth logic from route into controller

* Adjust scope check to default to no permission granted on existing endpoints for enhanced security.

* Support user generated tokens (not linked to a third party app)

* Add scopes endpoint that lists all available scopes.

* Support multiple scopes being required for an endpoint.

* Implement database scopes

* Implement all scope checks

* Support read, write, delete specific permissions.

* Code cleanup and minor bug fixes

* Ensure appDID is always optional

* Update to latest verida with legacy DID patch.

* Rename tests from oauth to auth

* Fix unit test not updated for modified auth response

* Correctly handle ignore scope check with missing scopes

* Fix issues with dependencies

* Remove expand scopes console output

* Fix scope expansion

* Add api scope to expanded scopes test

* Add support for resolved scopes endpoint

* Add resolved scopes route

* Change server context to be different from the Vault to avoid any database clashes

* Add permission properties to database and datastore controller endpoints, to ensure scopes are correctly checked with permissions

* Neatly handle invalid schema URLs

* Support blacklisting some data schemas and databases from scoped access

* Flag if a datastore schema is known or unknown in the resolved scopes

* Handle scope edge cases (invalid API scope), single scope not returned.

* Support a scope validity response from the validate scopes endpoint

* Feature/auth middleware (#219)

* Start of refactor to have auth middleware
* Implement datastore scope permissions. Continue refactor to use auth middleware. Add passing unit tests. Remove personal prompt. Revoke tokens after unit tests.
* Limit data access for keyword tool, agent prompt by scopes.
* Output available tools in agent response

* Name plural is now returned for resolved schema scopes

* Require session auth for connection routes

* Use named params for datastore endpoints

* Update database endpoints to use named parameters

* Fix named parameter issue on search/datastore route

* Update resolved scopes to include titlePlural

* Feature/221 app usage tracking (#222)

* Implement basic account / user connection logging and request logging.

* Support app DID's getting usage stats, requests etc.

* Avoid duplicate scopes when validating.

* Fix incorrect auth config on connection routes

* Correctly handle logging response length when body is not a string.

* Support schema URI in profile prompt endpoint (#211)

* feat: add support for schema URI in profile prompt endpoint
* feat: improve prompt of llm profile endpoint
* chore: remove console

* Fix database params

* Fix recordId params for database.getById()

* Feature/223 billing basics (#224)

* Untested basic billing

* Implement basic billing

* Start of VDA deposit page

* Add alchemy manager to get VDA price and fetch ERC20 token transfer details

* Support depositing crypto

* Support enabling / disabling billing

* Support caching VDA price

* Support payer in the auth request. Support billing with requests, with unit tests.

* Minor bug fixes

* Support getting and registering an account.  Support fetching vda price. Deposit bug fixes. Handle missing transaction nicely.

* Add DID to GET /auth/token response.

* Replace + with _ in API key to avoid query string issues. Improve invalid scope request error message.

* Fix datastore routes

* Support outputing credits in scope endpoint

* Update version number and changelog

* Update CHANGELOG

---------

Co-authored-by: Chris <chris@verida.io>
Co-authored-by: aurelticot <aurelticot.dev@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant