diff --git a/README.md b/README.md index 0bbc2843..53210ef7 100644 --- a/README.md +++ b/README.md @@ -1,18 +1,26 @@ -#

🦮 db-ally

+
-

- Efficient, consistent and secure library for querying structured data with natural language + + + dbally logo + + +
+
+ +

+ Efficient, consistent and secure library for querying structured data with natural language

---- +[![PyPI - License](https://img.shields.io/pypi/l/dbally)](https://pypi.org/project/dbally) +[![PyPI - Version](https://img.shields.io/pypi/v/dbally)](https://pypi.org/project/dbally) +[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/dbally)](https://pypi.org/project/dbally) -* **Documentation:** [db-ally.deepsense.ai](https://db-ally.deepsense.ai/) -* **Source code:** [github.com/deepsense-ai/db-ally](https://github.com/deepsense-ai/db-ally) +
--- - -**db-ally** is an LLM-powered library for creating natural language interfaces to data sources. While it occupies a similar space to the text-to-SQL solutions, its goals and methods are different. db-ally allows developers to outline specific use cases for the LLM to handle, detailing the desired data format and the possible operations to fetch this data. +db-ally is an LLM-powered library for creating natural language interfaces to data sources. While it occupies a similar space to the text-to-SQL solutions, its goals and methods are different. db-ally allows developers to outline specific use cases for the LLM to handle, detailing the desired data format and the possible operations to fetch this data. db-ally effectively shields the complexity of the underlying data source from the model, presenting only the essential information needed for solving the specific use cases. Instead of generating arbitrary SQL, the model is asked to generate responses in a simplified query language. diff --git a/docs/assets/banner-dark.svg b/docs/assets/banner-dark.svg new file mode 100644 index 00000000..3b580fe3 --- /dev/null +++ b/docs/assets/banner-dark.svg @@ -0,0 +1,266 @@ + + + + + + diff --git a/docs/assets/banner-light.svg b/docs/assets/banner-light.svg new file mode 100644 index 00000000..e0ef1d4d --- /dev/null +++ b/docs/assets/banner-light.svg @@ -0,0 +1,266 @@ + + + + + + diff --git a/docs/assets/favicon.ico b/docs/assets/favicon.ico new file mode 100644 index 00000000..7683ac51 Binary files /dev/null and b/docs/assets/favicon.ico differ diff --git a/docs/assets/guide_dog_lg.png b/docs/assets/guide_dog_lg.png deleted file mode 100644 index dee16c22..00000000 Binary files a/docs/assets/guide_dog_lg.png and /dev/null differ diff --git a/docs/assets/guide_dog_sm.png b/docs/assets/guide_dog_sm.png deleted file mode 100644 index 85f91ee5..00000000 Binary files a/docs/assets/guide_dog_sm.png and /dev/null differ diff --git a/docs/assets/logo.svg b/docs/assets/logo.svg new file mode 100644 index 00000000..5d4381d0 --- /dev/null +++ b/docs/assets/logo.svg @@ -0,0 +1,229 @@ + + + + diff --git a/docs/index.md b/docs/index.md index 1b5c6460..18f11b05 100644 --- a/docs/index.md +++ b/docs/index.md @@ -3,6 +3,117 @@ hide: - navigation --- +# db-ally docs + + + +
+ ![dbally logo](https://raw.githubusercontent.com/deepsense-ai/db-ally/mp/update-logo/docs/assets/banner-light.svg#only-light){ width="30%" } + ![dbally logo](https://raw.githubusercontent.com/deepsense-ai/db-ally/mp/update-logo/docs/assets/banner-dark.svg#only-dark){ width="30%" } +
+ +

+ Efficient, consistent and secure library for querying structured data with natural language +

+ +
+ + + PyPI - License + + + + PyPI - Version + + + + PyPI - Python Version + + +
+ --- ---8<-- "README.md" +db-ally is an LLM-powered library for creating natural language interfaces to data sources. While it occupies a similar space to the text-to-SQL solutions, its goals and methods are different. db-ally allows developers to outline specific use cases for the LLM to handle, detailing the desired data format and the possible operations to fetch this data. + +db-ally effectively shields the complexity of the underlying data source from the model, presenting only the essential information needed for solving the specific use cases. Instead of generating arbitrary SQL, the model is asked to generate responses in a simplified query language. + +The benefits of db-ally can be described in terms of its four main characteristics: + +* **Consistency**: db-ally ensures predictable output formats and confines operations to those predefined by developers, making it particularly well-suited for applications with precise requirements on their behavior or data format +* **Security**: db-ally prevents direct database access and arbitrary SQL execution, bolstering system safety +* **Efficiency**: db-ally hides most of the underlying database complexity, enabling the LLM to concentrate on essential aspects and improving performance +* **Portability**: db-ally introduces an abstraction layer between the model and the data, ensuring easy integration with various database technologies and other data sources. + +## Quickstart + +In db-ally, developers define their use cases by implementing [**views**](https://db-ally.deepsense.ai/concepts/views) and **filters**. A list of possible filters is presented to the LLM in terms of [**IQL**](https://db-ally.deepsense.ai/concepts/iql) (Intermediate Query Language). Views are grouped and registered within a [**collection**](https://db-ally.deepsense.ai/concepts/views), which then serves as an entry point for asking questions in natural language. + +This is a basic implementation of a db-ally view for an example HR application, which retrieves candidates from an SQL database: + +```python +from dbally import decorators, SqlAlchemyBaseView, create_collection +from dbally.llms.litellm import LiteLLM +from sqlalchemy import create_engine + +class CandidateView(SqlAlchemyBaseView): + """ + A view for retrieving candidates from the database. + """ + + def get_select(self): + """ + Defines which columns to select. + """ + return sqlalchemy.select(Candidate.id, Candidate.name, Candidate.country) + + @decorators.view_filter() + def from_country(self, country: str): + """ + Filter candidates from a specific country. + """ + return Candidate.country == country + +engine = create_engine('sqlite:///examples/recruiting/data/candidates.db') +llm = LiteLLM(model_name="gpt-3.5-turbo") +my_collection = create_collection("collection_name", llm) +my_collection.add(CandidateView, lambda: CandidateView(engine)) + +my_collection.ask("Find candidates from United States") +``` + +For a concrete step-by-step example on how to use db-ally, go to [Quickstart](https://db-ally.deepsense.ai/quickstart/) guide. For a more learning-oriented experience, check our db-ally [Tutorial](https://db-ally.deepsense.ai/tutorials/). + +## Motivation + +db-ally was originally developed at [deepsense.ai](https://deepsense.ai). In our work on various projects, we frequently encountered the need to retrieve data from data sources, typically databases, in response to natural language queries. + +The standard approach to this issue involves using the text-to-SQL technique. While this method is powerful, it is also complex and challenging to control. Often, the results were unsatisfactory because the Language Model lacked the necessary context to understand the specific requirements of the application and the business logic behind the data. + +This led us to experiment with a more structured approach. In this method, the developer defines the specific use cases for the Language Model to handle, detailing the desired data format and the possible operations to retrieve this data. This approach proved to be more efficient, predictable, and easier to manage, making it simpler to integrate with the rest of the system. + +Eventually, we decided to create a library that would allow us to use this approach in a more systematic way, and we made it open-source for the community. + +## Installation + +To install db-ally, execute the following command: + +```bash +pip install dbally +``` + +Additionally, you can install one of our extensions to use specific features. + +* `dbally[litellm]`: Use [100+ LLMs](https://docs.litellm.ai/docs/providers) +* `dbally[faiss]`: Use [Faiss](https://github.com/facebookresearch/faiss) indexes for similarity search +* `dbally[langsmith]`: Use [LangSmith](https://www.langchain.com/langsmith) for query tracking + +```bash +pip install dbally[litellm,faiss,langsmith] +``` + +## License + +db-ally is released under MIT license. diff --git a/docs/stylesheets/extra.css b/docs/stylesheets/extra.css index 698837d0..01232bf2 100644 --- a/docs/stylesheets/extra.css +++ b/docs/stylesheets/extra.css @@ -2,12 +2,6 @@ --md-primary-fg-color: #00b0e0; } -.md-header__button.md-logo { - margin: 0; - padding: 0; +.md-header__title { + margin-left: 0.5rem !important; } - -.md-header__button.md-logo img, .md-header__button.md-logo svg { - height: 1.8rem; - width: 1.8rem; -} \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index 0129b1ff..096954a7 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -1,12 +1,15 @@ -site_name: db-ally +site_name: db-ally docs +site_description: Efficient, consistent and secure library for querying structured data with natural language +site_url: https://db-ally.deepsense.ai +repo_name: deepsense-ai/db-ally repo_url: https://github.com/deepsense-ai/db-ally +copyright: Copyright © 2024 deepsense.ai nav: - - Home: index.md + - db-ally docs: index.md - Quickstart: - quickstart/index.md - quickstart/quickstart2.md - quickstart/quickstart3.md - - Tutorials: tutorials.md - Concepts: - concepts/views.md - concepts/structured_views.md @@ -37,6 +40,7 @@ nav: - how-to/trace_runs_with_otel.md - how-to/create_custom_event_handler.md - how-to/openai_assistants_integration.md + - Tutorials: tutorials.md - API Reference: - reference/index.md - reference/collection.md @@ -86,26 +90,31 @@ nav: - about/contact.md theme: name: material - logo: assets/guide_dog_lg.png - favicon: assets/guide_dog_sm.png + logo: assets/logo.svg + favicon: assets/favicon.ico icon: repo: fontawesome/brands/github palette: - - media: "(prefers-color-scheme: light)" + - media: "(prefers-color-scheme)" + toggle: + icon: material/lightbulb-auto + name: Switch to light mode + - media: '(prefers-color-scheme: light)' scheme: default primary: primary toggle: - icon: material/brightness-7 + icon: material/lightbulb name: Switch to dark mode - - media: "(prefers-color-scheme: dark)" + - media: '(prefers-color-scheme: dark)' scheme: slate primary: primary toggle: - icon: material/brightness-4 - name: Switch to light mode + icon: material/lightbulb-outline + name: Switch to system preference features: - navigation.footer - navigation.tabs + - navigation.tabs.sticky - navigation.top - content.code.annotate - content.code.copy @@ -125,6 +134,7 @@ markdown_extensions: - pymdownx.snippets - pymdownx.inlinehilite - attr_list + - md_in_html - pymdownx.details - def_list - pymdownx.tasklist: @@ -161,3 +171,29 @@ extra: analytics: provider: google property: G-FBBJRN0H0G + feedback: + title: Was this page helpful? + ratings: + - icon: material/emoticon-happy-outline + name: This page was helpful + data: 1 + note: >- + Thanks for your feedback! + - icon: material/emoticon-sad-outline + name: This page could be improved + data: 0 + note: >- + Thanks for your feedback! + social: + - icon: fontawesome/brands/github + link: https://github.com/deepsense-ai + - icon: fontawesome/brands/x-twitter + link: https://x.com/deepsense_ai + - icon: fontawesome/brands/linkedin + link: https://linkedin.com/company/deepsense-ai + - icon: fontawesome/brands/youtube + link: https://youtube.com/@deepsenseai + - icon: fontawesome/brands/medium + link: https://medium.com/deepsense-ai + - icon: fontawesome/solid/globe + link: https://deepsense.ai