Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(benchmarks): aggregation benchmarks #84

Merged
merged 29 commits into from
Sep 25, 2024
Merged

feat(benchmarks): aggregation benchmarks #84

merged 29 commits into from
Sep 25, 2024

Conversation

micpst
Copy link
Collaborator

@micpst micpst commented Aug 19, 2024 •
edited
Loading

This PR adds missing aggregation benchmarks

PatrykWyzgowski and others added 20 commits July 12, 2024 16:40
@PatrykWyzgowski
Adding aggregation handling for SqlAlchemyBaseView extending quicksta…
6d66b00 
…rt example.
@PatrykWyzgowski
Applying initial review feedback. Adding both filters and aggregation…
be12e6e 
… to result context.
@PatrykWyzgowski
Renaming subquery attribute and method argument to filtered_query
33d5b2e
@PatrykWyzgowski
Simplified question to the model.
2e77fbc
@PatrykWyzgowski
Fixing unnecessary-pass.
41e88ed
@PatrykWyzgowski
Continuation of review feedback application.
dfe3e13
@PatrykWyzgowski
Adjusting filter prompt not to mix IQL with 'UNSUPPORTED QUERY'. Furt…
c09e68e 
…her review feedback application.
@PatrykWyzgowski
Applied changes suggested in a comment related to Aggregations not ge…
c6bbf90 
…nerated by IQLGenerator (separate formatter implemented).
@PatrykWyzgowski
Applying pre-commit hooks.
4765dde
@PatrykWyzgowski
Mocking AggregationFormat in tests.
2918ba5
@PatrykWyzgowski
Merge branch 'main' into pw/add-single-aggregation
0b7e50a
@PatrykWyzgowski
Mocking methods of the view related to aggregations to make them comp…
5511729 
…liant to tests.
@PatrykWyzgowski
Pre-commit fixes.
ae26c8b
@micpst
merge main
63c3adc
@micpst
revert to prev approach
a2169f2
@micpst
fix tests
013cb69
@micpst
add more tests
f0a2f6e
@micpst
trying to fix tests (localy working)
aeb6295
@micpst
fix tests for python 3.8
d21f4e1
@micpst
update views
e473cc4
@micpst micpst added the feature New feature or request label Aug 19, 2024
@micpst micpst self-assigned this Aug 19, 2024
@github-actions GitHub Actions
Copy link

Trivy scanning results.

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 19, 2024 •
edited
Loading

badge

Code Coverage Summary

Filename                                                  Stmts    Miss  Cover    Missing
------------------------------------------------------  -------  ------  -------  ---------------------------------------------------------------------
dbally/_main.py                                              13       1  92.31%   10
dbally/_types.py                                              8       1  87.50%   24
dbally/exceptions.py                                          1       0  100.00%
dbally/assistants/base.py                                    24       0  100.00%
dbally/assistants/openai.py                                  59       2  96.61%   59-76
dbally/audit/event_tracker.py                                36       8  77.78%   38-40, 53, 64, 74, 91, 97
dbally/audit/events.py                                       34       0  100.00%
dbally/audit/spans.py                                         7       0  100.00%
dbally/audit/event_handlers/base.py                          15       0  100.00%
dbally/audit/event_handlers/buffer_event_handler.py           8       0  100.00%
dbally/audit/event_handlers/cli_event_handler.py             56      35  37.50%   11-13, 47-55, 65-66, 79-98, 120-127, 138-145
dbally/audit/event_handlers/langsmith_event_handler.py       29      25  13.79%   6-106
dbally/audit/event_handlers/otel_event_handler.py            74      27  63.51%   19, 123, 126, 138-139, 159-179, 191-199, 209-219, 222-223
dbally/collection/collection.py                             126       3  97.62%   136, 153, 325
dbally/collection/exceptions.py                              13       0  100.00%
dbally/collection/results.py                                 14       0  100.00%
dbally/embeddings/base.py                                     5       0  100.00%
dbally/embeddings/exceptions.py                              15       6  60.00%   10-11, 20, 29-30, 39
dbally/embeddings/litellm.py                                 28      12  57.14%   7-8, 44, 68-84
dbally/gradio/interface.py                                  104     104  0.00%    1-362
dbally/iql/_exceptions.py                                    49       1  97.96%   74
dbally/iql/_processor.py                                     94       5  94.68%   21, 82, 97, 170, 176
dbally/iql/_query.py                                         24       1  95.83%   9
dbally/iql/_type_validators.py                               38       2  94.74%   24, 28
dbally/iql/syntax.py                                         36       9  75.00%   6-9, 27, 36, 60, 63-66
dbally/iql_generator/iql_generator.py                        69       5  92.75%   247-249, 315-316
dbally/iql_generator/prompt.py                               33       2  93.94%   40, 69
dbally/llms/base.py                                          28       1  96.43%   34
dbally/llms/litellm.py                                       23      10  56.52%   7-8, 47-53, 60, 77
dbally/llms/local.py                                         18      18  0.00%    1-60
dbally/llms/clients/base.py                                  23       2  91.30%   46-47
dbally/llms/clients/exceptions.py                            15       6  60.00%   10-11, 20, 29-30, 39
dbally/llms/clients/litellm.py                               44      21  52.27%   8-9, 65-71, 97-120
dbally/llms/clients/local.py                                 33      33  0.00%    1-95
dbally/nl_responder/nl_responder.py                          24       4  83.33%   74-85
dbally/nl_responder/prompts.py                               19       4  78.95%   64-67
dbally/prompt/elements.py                                    25       1  96.00%   29
dbally/prompt/template.py                                    65       8  87.69%   33, 41, 49, 96, 110, 127, 153, 205
dbally/similarity/chroma_store.py                            37       0  100.00%
dbally/similarity/elastic_vector_search.py                   19      16  15.79%   5-102
dbally/similarity/elasticsearch_store.py                     22      19  13.64%   5-107
dbally/similarity/faiss_store.py                             38      35  7.90%    5-103
dbally/similarity/fetcher.py                                  5       0  100.00%
dbally/similarity/index.py                                   30       2  93.33%   26, 70
dbally/similarity/sqlalchemy_base.py                         44      19  56.82%   35-37, 46, 68, 77, 86-89, 99-105, 123-126
dbally/similarity/store.py                                    7       0  100.00%
dbally/view_selection/base.py                                 7       0  100.00%
dbally/view_selection/llm_view_selector.py                   17       0  100.00%
dbally/view_selection/prompt.py                               9       0  100.00%
dbally/view_selection/random_view_selector.py                10      10  0.00%    1-36
dbally/views/base.py                                         16       1  93.75%   52
dbally/views/decorators.py                                   11       0  100.00%
dbally/views/exceptions.py                                    7       3  57.14%   20-22
dbally/views/exposed_functions.py                            35       3  91.43%   26-29
dbally/views/methods_base.py                                 41       2  95.12%   85, 93
dbally/views/pandas_base.py                                  52       1  98.08%   97
dbally/views/sqlalchemy_base.py                              40       7  82.50%   64, 79-82, 99-103
dbally/views/structured.py                                   49       1  97.96%   77
dbally/views/freeform/text2sql/config.py                     21       1  95.24%   47
dbally/views/freeform/text2sql/exceptions.py                  7       3  57.14%   12-14
dbally/views/freeform/text2sql/prompt.py                     11       0  100.00%
dbally/views/freeform/text2sql/view.py                       93      22  76.34%   49, 52, 55, 58, 61, 73, 151, 155-158, 161, 191, 206-207, 215, 228-233
dbally_cli/main.py                                            5       5  0.00%    1-13
dbally_cli/text2sql.py                                       94      94  0.00%    1-248
dbally_codegen/autodiscovery.py                             122      18  85.25%   54-57, 241-243, 264-278, 281-284, 353-354, 450-455
dbally_codegen/generator.py                                 175       7  96.00%   81, 91, 314, 342, 360, 374, 420
TOTAL                                                      2353     626  73.40%

Diff against main

Filename      Stmts    Miss  Cover
----------  -------  ------  --------
TOTAL             0       0  +100.00%

Results for commit: 0193e4b

Minimum allowed coverage is 60%

♻️ This comment has been updated with latest results

@micpst micpst force-pushed the mp/aggregation-benchmarks branch from aaa3039 to e69fc07 Compare September 2, 2024 23:02
@micpst micpst marked this pull request as ready for review September 24, 2024 08:05
mhordynski
mhordynski approved these changes Sep 24, 2024
View reviewed changes
Copy link
Member

@mhordynski mhordynski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@micpst micpst merged commit 26033f7 into main Sep 25, 2024
3 checks passed
@micpst micpst deleted the mp/aggregation-benchmarks branch September 25, 2024 07:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mhordynski mhordynski mhordynski approved these changes

Assignees

@micpst micpst

Labels
feature New feature or request
Projects
dbally 1.0
Status: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@micpst @mhordynski @PatrykWyzgowski