-
Notifications
You must be signed in to change notification settings - Fork 291
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature/include vectors option document chunks #1419
Merged
emrgnt-cmplxty
merged 16 commits into
dev-minor
from
feature/include-vectors-option-document-chunks
Oct 16, 2024
Merged
Feature/include vectors option document chunks #1419
emrgnt-cmplxty
merged 16 commits into
dev-minor
from
feature/include-vectors-option-document-chunks
Oct 16, 2024
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Fix async JSON parsing * Remove score completion from js * clean up js * lockfile
* building a custom logger for r2r * fix log * maintain bkwd compat
* add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 Looks good to me! Reviewed everything up to 1c61f86 in 1 minute and 21 seconds
More details
- Looked at
306
lines of code in10
files - Skipped
0
files when reviewing. - Skipped posting
3
drafted comments based on config settings.
1. py/core/providers/database/vector.py:508
- Draft comment:
Consider adding a comment to explain the purpose of theinclude_vectors
parameter and its impact on the query performance and data transfer. - Reason this comment was not posted:
Confidence changes required:50%
The addition of theinclude_vectors
parameter in theget_document_chunks
method is a significant change. It allows users to optionally include vectors in the response, which can be useful for applications that need to process or analyze these vectors. However, the implementation should ensure that the vectors are only included when necessary to avoid unnecessary data transfer and processing.
2. py/core/main/api/management_router.py:370
- Draft comment:
Ensure that theinclude_vectors
parameter is consistently handled across all methods that support it, and update the documentation to reflect this new feature. - Reason this comment was not posted:
Confidence changes required:50%
Theinclude_vectors
parameter is added to multiple methods across different files. This parameter allows users to include vectors in the response, which is a useful feature for applications that need to process these vectors. However, it's important to ensure that this parameter is consistently handled across all methods and that the documentation reflects this new feature.
3. py/sdk/management.py:268
- Draft comment:
Ensure that theinclude_vectors
parameter is consistently handled across all methods that support it, and update the documentation to reflect this new feature. - Reason this comment was not posted:
Confidence changes required:50%
Theinclude_vectors
parameter is added to thedocument_chunks
method in the SDK. This parameter allows users to include vectors in the response, which is a useful feature for applications that need to process these vectors. However, it's important to ensure that this parameter is consistently handled across all methods and that the documentation reflects this new feature.
Workflow ID: wflow_0EUrxLP7sHdZFd6l
You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet
mode, and more.
emrgnt-cmplxty
added a commit
that referenced
this pull request
Oct 17, 2024
* Feature/include vectors option document chunks (#1419) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * add option to include vectors in document chunks * checkin * update vector --------- Co-authored-by: Nolan Tremelling <[email protected]> * Allow env var to set the default R2R deployment for the dashboard (#1417) * Feature/various documentation tweaks (#1422) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * add option to include vectors in document chunks * checkin * update vector * some various documentation tweaks * some various documentation tweaks --------- Co-authored-by: Nolan Tremelling <[email protected]> * Graphrag tests (#1418) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * up * add tests --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> * Modify graphrag tests timeouts (#1416) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * modify timeouts --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> * feat: Make prompt provider methods asynchronous (comments below) (#1415) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * feat: Make prompt provider methods asynchronous --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> * bump pyproject version --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: Shreyas Pimpalgaonkar <[email protected]>
emrgnt-cmplxty
added a commit
that referenced
this pull request
Oct 18, 2024
* Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * up * add tests * Feature/include vectors option document chunks (#1419) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * add option to include vectors in document chunks * checkin * update vector --------- Co-authored-by: Nolan Tremelling <[email protected]> * Allow env var to set the default R2R deployment for the dashboard (#1417) * modify community_summary_prompt function and corresponding prompt * add tests * up * Feature/various documentation tweaks (#1422) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * add option to include vectors in document chunks * checkin * update vector * some various documentation tweaks * some various documentation tweaks --------- Co-authored-by: Nolan Tremelling <[email protected]> * Graphrag tests (#1418) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * up * add tests --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> * Modify graphrag tests timeouts (#1416) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * modify timeouts --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> * feat: Make prompt provider methods asynchronous (comments below) (#1415) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * feat: Make prompt provider methods asynchronous --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> * up * up --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]>
shreyaspimpalgaonkar
added a commit
that referenced
this pull request
Oct 18, 2024
* Modify graphrag prompt (#1421) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * up * add tests * Feature/include vectors option document chunks (#1419) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * add option to include vectors in document chunks * checkin * update vector --------- Co-authored-by: Nolan Tremelling <[email protected]> * Allow env var to set the default R2R deployment for the dashboard (#1417) * modify community_summary_prompt function and corresponding prompt * add tests * up * Feature/various documentation tweaks (#1422) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * add option to include vectors in document chunks * checkin * update vector * some various documentation tweaks * some various documentation tweaks --------- Co-authored-by: Nolan Tremelling <[email protected]> * Graphrag tests (#1418) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * up * add tests --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> * Modify graphrag tests timeouts (#1416) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * modify timeouts --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> * feat: Make prompt provider methods asynchronous (comments below) (#1415) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * feat: Make prompt provider methods asynchronous --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> * up * up --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> * Add routes, service level methods around conversations (#1420) * Add routes, service level methods around conversations * Slight refactor to match project conventions, add JS methods * Updated JS methods * JS docs * Add python * Update JS user tests * Bump JS, fix python SKD ingest chunks comments * Update pyproject.toml --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]>
emrgnt-cmplxty
added a commit
that referenced
this pull request
Oct 20, 2024
* Modify graphrag prompt (#1421) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * up * add tests * Feature/include vectors option document chunks (#1419) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * add option to include vectors in document chunks * checkin * update vector --------- Co-authored-by: Nolan Tremelling <[email protected]> * Allow env var to set the default R2R deployment for the dashboard (#1417) * modify community_summary_prompt function and corresponding prompt * add tests * up * Feature/various documentation tweaks (#1422) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * add option to include vectors in document chunks * checkin * update vector * some various documentation tweaks * some various documentation tweaks --------- Co-authored-by: Nolan Tremelling <[email protected]> * Graphrag tests (#1418) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * up * add tests --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> * Modify graphrag tests timeouts (#1416) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * modify timeouts --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> * feat: Make prompt provider methods asynchronous (comments below) (#1415) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * feat: Make prompt provider methods asynchronous --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> * up * up --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> * Add routes, service level methods around conversations (#1420) * Add routes, service level methods around conversations * Slight refactor to match project conventions, add JS methods * Updated JS methods * JS docs * Add python * Update JS user tests * add deduplication pipe, workflow, api, sdk, cli * add summary workflow * bug fixes * pre-commit * working * search working * adding dedup test files * modify the update query * precommit * more testing --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]>
emrgnt-cmplxty
added a commit
that referenced
this pull request
Oct 21, 2024
* Feature/include vectors option document chunks (#1419) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * add option to include vectors in document chunks * checkin * update vector --------- Co-authored-by: Nolan Tremelling <[email protected]> * Allow env var to set the default R2R deployment for the dashboard (#1417) * Feature/various documentation tweaks (#1422) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * add option to include vectors in document chunks * checkin * update vector * some various documentation tweaks * some various documentation tweaks --------- Co-authored-by: Nolan Tremelling <[email protected]> * Graphrag tests (#1418) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * up * add tests --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> * Modify graphrag tests timeouts (#1416) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * modify timeouts --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> * feat: Make prompt provider methods asynchronous (comments below) (#1415) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * feat: Make prompt provider methods asynchronous --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> * bump pyproject version * first commit * towards slimmer vector implementation logic * up * iterate * up * checkin * up * work doc chunks * working vector search * working full text search * remove asyncpg * passing vector tests * up * merge * rm pytest * up * up * fix delete * up * up --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: Shreyas Pimpalgaonkar <[email protected]>
emrgnt-cmplxty
added a commit
that referenced
this pull request
Oct 23, 2024
* fix-actions (#1426) * up * modify * add to github path * Contextual Chunk Enrichment (#1433) * add semantic chunking * working * precommit * pre-commits * Entity Deduplication (#1431) * Modify graphrag prompt (#1421) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * up * add tests * Feature/include vectors option document chunks (#1419) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * add option to include vectors in document chunks * checkin * update vector --------- Co-authored-by: Nolan Tremelling <[email protected]> * Allow env var to set the default R2R deployment for the dashboard (#1417) * modify community_summary_prompt function and corresponding prompt * add tests * up * Feature/various documentation tweaks (#1422) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * add option to include vectors in document chunks * checkin * update vector * some various documentation tweaks * some various documentation tweaks --------- Co-authored-by: Nolan Tremelling <[email protected]> * Graphrag tests (#1418) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * up * add tests --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> * Modify graphrag tests timeouts (#1416) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * modify timeouts --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> * feat: Make prompt provider methods asynchronous (comments below) (#1415) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * feat: Make prompt provider methods asynchronous --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> * up * up --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> * Add routes, service level methods around conversations (#1420) * Add routes, service level methods around conversations * Slight refactor to match project conventions, add JS methods * Updated JS methods * JS docs * Add python * Update JS user tests * add deduplication pipe, workflow, api, sdk, cli * add summary workflow * bug fixes * pre-commit * working * search working * adding dedup test files * modify the update query * precommit * more testing --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> * Refactor Python SDK for Intellisense, Thread Safety (#1430) * Refactor Python SDK * Fix CLI after SDK changes * Add convo to agent * Update conversation error handling, JS * Remove unused, bad import * Feature/cleanup refactor vector collection (#1432) * Feature/include vectors option document chunks (#1419) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * add option to include vectors in document chunks * checkin * update vector --------- Co-authored-by: Nolan Tremelling <[email protected]> * Allow env var to set the default R2R deployment for the dashboard (#1417) * Feature/various documentation tweaks (#1422) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * add option to include vectors in document chunks * checkin * update vector * some various documentation tweaks * some various documentation tweaks --------- Co-authored-by: Nolan Tremelling <[email protected]> * Graphrag tests (#1418) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * up * add tests --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> * Modify graphrag tests timeouts (#1416) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * modify timeouts --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> * feat: Make prompt provider methods asynchronous (comments below) (#1415) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * feat: Make prompt provider methods asynchronous --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> * bump pyproject version * first commit * towards slimmer vector implementation logic * up * iterate * up * checkin * up * work doc chunks * working vector search * working full text search * remove asyncpg * passing vector tests * up * merge * rm pytest * up * up * fix delete * up * up --------- Co-authored-by: Nolan Tremelling <[email protected]> Co-authored-by: Shreyas Pimpalgaonkar <[email protected]> * Add tests + Cleanup (#1437) * up * add tests * test rename to sdk * up * fix tests * typo * modify chunk enrichment prompt (#1438) * modify prompt * up * Fix type error on port argument of CLI (#1439) * finish (#1440) * finish * up * fix * fix * up * fix * final cleanups * fix naming convention * fix schema error * increase timeout * split graphrag actions * fix collection exists error * up (#1442) * Add error message (#1443) * up * sdk fix * locally testing build * up docs (#1445) * checkin work (#1444) * checkin work * finish index functionality extension * fix concurrency * add alembic (#1446) * Prompt Tuning (#1447) * Check in * Fix after merging dev-minor in * Ensure to not cause int overflow with hatchet (#1454) * Bump JS (#1456) * Ensure to not cause int overflow with hatchet * bump js * improve migration implementation (#1452) * improve migration implementation * refine migrations to include kg * add alembic cli * extend documentation * extend docs and all that * Revert change of default behaviour of entities endpoint, docs, tests (#1455) * change def behavior of entities + delete endpoint * pre-commit * add deduplication tests * Delete graph (#1450) * up docs * up * up * rename to raw_chunks * up * add tests * up * up * change default * change cli * separate out deduplication tests * change run type in the test * up * up * add test concurrency * up * rm concurrency groups * rm dedup tests * remove json * tests * up * fix lock * Update postgres.py * Feature/merge dev minor main (#1457) * add run without orchestration (#1448) * add run without orchestration * bump versions * bump versions * bump versions * fix * up * add end points * add run without orchestration (#1448) (#1458) * add run without orchestration * bump versions * bump versions * bump versions * fix * up * sync migration changes * Ensure that we await ingest files in ingest_files method (#1460) * Nolan/await update files (#1461) * Ensure that we await ingest files in ingest_files method * Await update files as well * Docs changes (#1462) * up * up * up * up * fix failed find and replace (#1463) * fix failed find and replace * fix * Fix JS Client for Ingest Chunks (#1464) * Ensure that we await ingest files in ingest_files method * Await update files as well * Fix js client --------- Co-authored-by: Shreyas Pimpalgaonkar <[email protected]> Co-authored-by: Nolan Tremelling <[email protected]>
emrgnt-cmplxty
deleted the
feature/include-vectors-option-document-chunks
branch
October 25, 2024 18:20
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Important
Add
include_vectors
option to document chunk retrieval in CLI, API, SDKs, and update documentation accordingly.--include-vectors
option todocument_chunks
command inmanagement.py
.include_vectors
query parameter todocument_chunks_app
inmanagement_router.py
.document_chunks
method inmanagement_service.py
to handleinclude_vectors
.get_document_chunks
invector.py
to supportinclude_vectors
.include_vectors
parameter todocument_chunks
method inmanagement.py
for Python and JS SDKs.include_vectors
option for document chunks.This description was created by for 1c61f86. It will automatically update as commits are pushed.