Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

complete logging port #1499

Merged
merged 1 commit into from
Oct 25, 2024
Merged

complete logging port #1499

merged 1 commit into from
Oct 25, 2024

Conversation

emrgnt-cmplxty
Copy link
Contributor

@emrgnt-cmplxty emrgnt-cmplxty commented Oct 25, 2024

Important

Refactor logging system by renaming modules, introducing PostgresLoggingHandler, and updating references across the codebase.

  • Logging Refactor:
    • Rename py/core/base/logging to py/core/base/logger and py/core/providers/logging to py/core/providers/logger.
    • Introduce PostgresLoggingHandler in database/logging.py for handling logging with PostgreSQL.
    • Update imports and references across multiple files to use the new logger module.
  • New Features:
    • Add LoggingHandler abstract class and implement PostgresLoggingHandler.
    • Implement conversation and message management in PostgresLoggingHandler.
  • Code Cleanup:
    • Remove deprecated RunType from logging/base.py.
    • Update conftest.py and test files to align with new logging structure.
  • Miscellaneous:
    • Fix minor issues and improve code consistency across the codebase.

This description was created by Ellipsis for 4255dc6. It will automatically update as commits are pushed.

@emrgnt-cmplxty emrgnt-cmplxty marked this pull request as ready for review October 25, 2024 22:17
@emrgnt-cmplxty emrgnt-cmplxty merged commit d4a8047 into dev Oct 25, 2024
8 of 22 checks passed
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Reviewed everything up to 4255dc6 in 50 seconds

More details
  • Looked at 2274 lines of code in 52 files
  • Skipped 0 files when reviewing.
  • Skipped posting 3 drafted comments based on config settings.
1. py/core/main/api/auth_router.py:19
  • Draft comment:
    The import statement for RunType is redundant because it is already imported from core.base.logger.base.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The import statement for RunType in py/core/main/api/auth_router.py is redundant because it is already imported from core.base.logger.base.
2. py/core/main/api/retrieval_router.py:20
  • Draft comment:
    The import statement for RunType is redundant because it is already imported from core.base.logger.base.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The import statement for RunType in py/core/main/api/retrieval_router.py is redundant because it is already imported from core.base.logger.base.
3. py/core/main/api/kg_router.py:6
  • Draft comment:
    The import statement for RunType is redundant because it is already imported from core.base.logger.base.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The import statement for RunType in py/core/main/api/kg_router.py is redundant because it is already imported from core.base.logger.base.

Workflow ID: wflow_Ldg4ZsliEDZ4Oi6h


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

emrgnt-cmplxty added a commit that referenced this pull request Oct 25, 2024
emrgnt-cmplxty added a commit that referenced this pull request Oct 25, 2024
* Feature/move logging (#1493)

* move logging provider out

* move logging provider to own directory, remove singleton

* cleanup

* Update js package (#1498)

* fix refactoring tweak (#1496)

* Fix JSON serialization and Prompt ID Bugs for Prompts (#1491)

* Bug in get prompts

* Add tests

* Prevent verbose logging on standup

* Remove kg as required key in config, await get_all_prompts

* Remove reference to fragment id

* comment out ingestion

* complete logging port (#1499)

---------

Co-authored-by: Nolan Tremelling <[email protected]>
emrgnt-cmplxty added a commit that referenced this pull request Oct 26, 2024
emrgnt-cmplxty added a commit that referenced this pull request Oct 26, 2024
* fix refactoring tweak (#1496)

* Fix JSON serialization and Prompt ID Bugs for Prompts (#1491)

* Bug in get prompts

* Add tests

* Prevent verbose logging on standup

* Remove kg as required key in config, await get_all_prompts

* Remove reference to fragment id

* comment out ingestion

* complete logging port (#1499)

* Fix handling for R2R exceptions (#1501)

* fix doc test (#1502)

* up

---------

Co-authored-by: Nolan Tremelling <[email protected]>
emrgnt-cmplxty added a commit that referenced this pull request Oct 28, 2024
* Feature/move logging (#1492)

* move logging provider out

* move logging provider to own directory, remove singleton

* cleanup

* fix refactoring tweak (#1496)

* Fix JSON serialization and Prompt ID Bugs for Prompts (#1491)

* Bug in get prompts

* Add tests

* Prevent verbose logging on standup

* Remove kg as required key in config, await get_all_prompts

* Remove reference to fragment id

* comment out ingestion

* complete logging port (#1499)

* Feature/dev rebased (#1500)

* Feature/move logging (#1493)

* move logging provider out

* move logging provider to own directory, remove singleton

* cleanup

* Update js package (#1498)

* fix refactoring tweak (#1496)

* Fix JSON serialization and Prompt ID Bugs for Prompts (#1491)

* Bug in get prompts

* Add tests

* Prevent verbose logging on standup

* Remove kg as required key in config, await get_all_prompts

* Remove reference to fragment id

* comment out ingestion

* complete logging port (#1499)

---------

Co-authored-by: Nolan Tremelling <[email protected]>

* Fix handling for R2R exceptions (#1501)

* fix doc test (#1502)

* Harmonize python SDK KG methods for optional params, add missing JS methods

---------

Co-authored-by: emrgnt-cmplxty <[email protected]>
Co-authored-by: emrgnt-cmplxty <[email protected]>
emrgnt-cmplxty added a commit that referenced this pull request Oct 30, 2024
* Feature/tweak actions (#1507)

* up

* tweak actions

* Sync JS SDK, Harmonize Python SDK KG Methods (#1511)

* Feature/move logging (#1492)

* move logging provider out

* move logging provider to own directory, remove singleton

* cleanup

* fix refactoring tweak (#1496)

* Fix JSON serialization and Prompt ID Bugs for Prompts (#1491)

* Bug in get prompts

* Add tests

* Prevent verbose logging on standup

* Remove kg as required key in config, await get_all_prompts

* Remove reference to fragment id

* comment out ingestion

* complete logging port (#1499)

* Feature/dev rebased (#1500)

* Feature/move logging (#1493)

* move logging provider out

* move logging provider to own directory, remove singleton

* cleanup

* Update js package (#1498)

* fix refactoring tweak (#1496)

* Fix JSON serialization and Prompt ID Bugs for Prompts (#1491)

* Bug in get prompts

* Add tests

* Prevent verbose logging on standup

* Remove kg as required key in config, await get_all_prompts

* Remove reference to fragment id

* comment out ingestion

* complete logging port (#1499)

---------

Co-authored-by: Nolan Tremelling <[email protected]>

* Fix handling for R2R exceptions (#1501)

* fix doc test (#1502)

* Harmonize python SDK KG methods for optional params, add missing JS methods

---------

Co-authored-by: emrgnt-cmplxty <[email protected]>
Co-authored-by: emrgnt-cmplxty <[email protected]>

* Clean up pagination and offset around KG (#1519)

* Move to R2R light for integration testing (#1521)

* Update community model

---------

Co-authored-by: emrgnt-cmplxty <[email protected]>
Co-authored-by: emrgnt-cmplxty <[email protected]>
emrgnt-cmplxty added a commit that referenced this pull request Oct 30, 2024
* Feature/tweak actions (#1507)

* up

* tweak actions

* Sync JS SDK, Harmonize Python SDK KG Methods (#1511)

* Feature/move logging (#1492)

* move logging provider out

* move logging provider to own directory, remove singleton

* cleanup

* fix refactoring tweak (#1496)

* Fix JSON serialization and Prompt ID Bugs for Prompts (#1491)

* Bug in get prompts

* Add tests

* Prevent verbose logging on standup

* Remove kg as required key in config, await get_all_prompts

* Remove reference to fragment id

* comment out ingestion

* complete logging port (#1499)

* Feature/dev rebased (#1500)

* Feature/move logging (#1493)

* move logging provider out

* move logging provider to own directory, remove singleton

* cleanup

* Update js package (#1498)

* fix refactoring tweak (#1496)

* Fix JSON serialization and Prompt ID Bugs for Prompts (#1491)

* Bug in get prompts

* Add tests

* Prevent verbose logging on standup

* Remove kg as required key in config, await get_all_prompts

* Remove reference to fragment id

* comment out ingestion

* complete logging port (#1499)

---------

Co-authored-by: Nolan Tremelling <[email protected]>

* Fix handling for R2R exceptions (#1501)

* fix doc test (#1502)

* Harmonize python SDK KG methods for optional params, add missing JS methods

---------

Co-authored-by: emrgnt-cmplxty <[email protected]>
Co-authored-by: emrgnt-cmplxty <[email protected]>

* Clean up pagination and offset around KG (#1519)

* Move to R2R light for integration testing (#1521)

---------

Co-authored-by: Nolan Tremelling <[email protected]>
emrgnt-cmplxty added a commit that referenced this pull request Oct 30, 2024
* Feature/tweak actions (#1507)

* up

* tweak actions

* Sync JS SDK, Harmonize Python SDK KG Methods (#1511)

* Feature/move logging (#1492)

* move logging provider out

* move logging provider to own directory, remove singleton

* cleanup

* fix refactoring tweak (#1496)

* Fix JSON serialization and Prompt ID Bugs for Prompts (#1491)

* Bug in get prompts

* Add tests

* Prevent verbose logging on standup

* Remove kg as required key in config, await get_all_prompts

* Remove reference to fragment id

* comment out ingestion

* complete logging port (#1499)

* Feature/dev rebased (#1500)

* Feature/move logging (#1493)

* move logging provider out

* move logging provider to own directory, remove singleton

* cleanup

* Update js package (#1498)

* fix refactoring tweak (#1496)

* Fix JSON serialization and Prompt ID Bugs for Prompts (#1491)

* Bug in get prompts

* Add tests

* Prevent verbose logging on standup

* Remove kg as required key in config, await get_all_prompts

* Remove reference to fragment id

* comment out ingestion

* complete logging port (#1499)

---------

Co-authored-by: Nolan Tremelling <[email protected]>

* Fix handling for R2R exceptions (#1501)

* fix doc test (#1502)

* Harmonize python SDK KG methods for optional params, add missing JS methods

---------

Co-authored-by: emrgnt-cmplxty <[email protected]>
Co-authored-by: emrgnt-cmplxty <[email protected]>

* Clean up pagination and offset around KG (#1519)

* Move to R2R light for integration testing (#1521)

* fix ollama pdf parser

---------

Co-authored-by: Nolan Tremelling <[email protected]>
emrgnt-cmplxty added a commit that referenced this pull request Oct 30, 2024
* adding bin support and make it default (#1508)

* Feature/tweak actions (#1507)

* up

* tweak actions

* adding bin sup and making it default

* tested and vetted

* up (#1510)

* up

* set verification to default false

* cleanup (#1512)

* cleanup

* cleanup prompt mgmt

* up

* cleanup printout

* cleanup new parser logic, set vlm as default for all providers

* allow user to re-override

* modify exp backoff implementation (#1513)

* Feature/tweak actions (#1507)

* up

* tweak actions

* modify exp backoff impl

---------

Co-authored-by: emrgnt-cmplxty <[email protected]>

* Patch/touchups (#1515)

* cleanup

* cleanup prompt mgmt

* up

* cleanup printout

* cleanup new parser logic, set vlm as default for all providers

* allow user to re-override

* add touchups

* add extra parsers (#1516)

* add extra parsers (#1518)

* minor fixes (#1514)

* Feature/add back ollama provider (#1522)

* add extra parsers

* add back ollama

* rvert auth workflow

* Feature/add prompt tests and cleanup (#1523)

* add extra parsers

* add prompt tests, cleanup

* add prompt tests, cleanup

* merge

* set mock console as default

* set mock console as default

* fix config

* Update community model (#1524)

* Feature/tweak actions (#1507)

* up

* tweak actions

* Sync JS SDK, Harmonize Python SDK KG Methods (#1511)

* Feature/move logging (#1492)

* move logging provider out

* move logging provider to own directory, remove singleton

* cleanup

* fix refactoring tweak (#1496)

* Fix JSON serialization and Prompt ID Bugs for Prompts (#1491)

* Bug in get prompts

* Add tests

* Prevent verbose logging on standup

* Remove kg as required key in config, await get_all_prompts

* Remove reference to fragment id

* comment out ingestion

* complete logging port (#1499)

* Feature/dev rebased (#1500)

* Feature/move logging (#1493)

* move logging provider out

* move logging provider to own directory, remove singleton

* cleanup

* Update js package (#1498)

* fix refactoring tweak (#1496)

* Fix JSON serialization and Prompt ID Bugs for Prompts (#1491)

* Bug in get prompts

* Add tests

* Prevent verbose logging on standup

* Remove kg as required key in config, await get_all_prompts

* Remove reference to fragment id

* comment out ingestion

* complete logging port (#1499)

---------

Co-authored-by: Nolan Tremelling <[email protected]>

* Fix handling for R2R exceptions (#1501)

* fix doc test (#1502)

* Harmonize python SDK KG methods for optional params, add missing JS methods

---------

Co-authored-by: emrgnt-cmplxty <[email protected]>
Co-authored-by: emrgnt-cmplxty <[email protected]>

* Clean up pagination and offset around KG (#1519)

* Move to R2R light for integration testing (#1521)

* Update community model

---------

Co-authored-by: emrgnt-cmplxty <[email protected]>
Co-authored-by: emrgnt-cmplxty <[email protected]>

* Patch/fix import bleed (#1526)

* Feature/tweak actions (#1507)

* up

* tweak actions

* Sync JS SDK, Harmonize Python SDK KG Methods (#1511)

* Feature/move logging (#1492)

* move logging provider out

* move logging provider to own directory, remove singleton

* cleanup

* fix refactoring tweak (#1496)

* Fix JSON serialization and Prompt ID Bugs for Prompts (#1491)

* Bug in get prompts

* Add tests

* Prevent verbose logging on standup

* Remove kg as required key in config, await get_all_prompts

* Remove reference to fragment id

* comment out ingestion

* complete logging port (#1499)

* Feature/dev rebased (#1500)

* Feature/move logging (#1493)

* move logging provider out

* move logging provider to own directory, remove singleton

* cleanup

* Update js package (#1498)

* fix refactoring tweak (#1496)

* Fix JSON serialization and Prompt ID Bugs for Prompts (#1491)

* Bug in get prompts

* Add tests

* Prevent verbose logging on standup

* Remove kg as required key in config, await get_all_prompts

* Remove reference to fragment id

* comment out ingestion

* complete logging port (#1499)

---------

Co-authored-by: Nolan Tremelling <[email protected]>

* Fix handling for R2R exceptions (#1501)

* fix doc test (#1502)

* Harmonize python SDK KG methods for optional params, add missing JS methods

---------

Co-authored-by: emrgnt-cmplxty <[email protected]>
Co-authored-by: emrgnt-cmplxty <[email protected]>

* Clean up pagination and offset around KG (#1519)

* Move to R2R light for integration testing (#1521)

---------

Co-authored-by: Nolan Tremelling <[email protected]>

* Patch/fix import bleed (#1527)

* Feature/tweak actions (#1507)

* up

* tweak actions

* Sync JS SDK, Harmonize Python SDK KG Methods (#1511)

* Feature/move logging (#1492)

* move logging provider out

* move logging provider to own directory, remove singleton

* cleanup

* fix refactoring tweak (#1496)

* Fix JSON serialization and Prompt ID Bugs for Prompts (#1491)

* Bug in get prompts

* Add tests

* Prevent verbose logging on standup

* Remove kg as required key in config, await get_all_prompts

* Remove reference to fragment id

* comment out ingestion

* complete logging port (#1499)

* Feature/dev rebased (#1500)

* Feature/move logging (#1493)

* move logging provider out

* move logging provider to own directory, remove singleton

* cleanup

* Update js package (#1498)

* fix refactoring tweak (#1496)

* Fix JSON serialization and Prompt ID Bugs for Prompts (#1491)

* Bug in get prompts

* Add tests

* Prevent verbose logging on standup

* Remove kg as required key in config, await get_all_prompts

* Remove reference to fragment id

* comment out ingestion

* complete logging port (#1499)

---------

Co-authored-by: Nolan Tremelling <[email protected]>

* Fix handling for R2R exceptions (#1501)

* fix doc test (#1502)

* Harmonize python SDK KG methods for optional params, add missing JS methods

---------

Co-authored-by: emrgnt-cmplxty <[email protected]>
Co-authored-by: emrgnt-cmplxty <[email protected]>

* Clean up pagination and offset around KG (#1519)

* Move to R2R light for integration testing (#1521)

* fix ollama pdf parser

---------

Co-authored-by: Nolan Tremelling <[email protected]>

* email auth false and js bump

* fix actions (#1528)

* Feature/add poppler check and fallback (#1529)

* fix actions

* fallback

* Patch/import shutil (#1530)

* fix actions

* fallback

* import shutil

* Feature/include basic pdf parsing everywhere (#1531)

* fix actions

* fallback

* import shutil

* add basic pdf as extra parser in all configs

* Remove non existent user login?

* attempt login

* Change password back

* add explicit setting, trigger rebuild

---------

Co-authored-by: Shreyas Pimpalgaonkar <[email protected]>
Co-authored-by: Nolan Tremelling <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant