From 8d3bb2d5845cf2d09dbe27621273b777236f60d5 Mon Sep 17 00:00:00 2001 From: Janaka Abeywardhana Date: Fri, 15 Mar 2024 21:27:56 +0000 Subject: [PATCH 1/4] docs: deployment and arch docs update --- docs/overview/deployment-scenarios.md | 28 +++++++++++++-------------- docs/overview/design-principles.md | 2 +- 2 files changed, 14 insertions(+), 16 deletions(-) diff --git a/docs/overview/deployment-scenarios.md b/docs/overview/deployment-scenarios.md index b9bce06f..1969adfb 100644 --- a/docs/overview/deployment-scenarios.md +++ b/docs/overview/deployment-scenarios.md @@ -1,27 +1,21 @@ +Docq can be deployed in several configurations trading off data isolation for operational convenience. -Docq is a **self-hosted, turnkey** software which means **for data security and privacy reasons we, the Docq Team does NOT offer a hosted service** because we believe that the best place to operate Docq is within your existing cloud infrastructure setup. +- **Isolated In Your Data centre**: Deploy on-prem on servers in your data centre. The most isolated hence most secure deployment. All your data stays within your data centre and enterprise network. LLMs will also have to be deployed on-premise to benefit from this option. We are happy to support you with a bespoke setup. **Bring LLMs to your data.** -It also lets a business adopting Docq operate exactly in the same way as they currently operate their IT infrastructure. You know best about your own operations. +- **Isolated In Your Cloud**: Deploy in your organisation's cloud account (Azure, AWS, GCP etc.). LLMs will need to be self-hosted in your cloud account. Similar to on-premise hosting, your data remains within network boundaries you control. **Bring LLMs to your data.** -Docq mainly caters for two deployment scenarios: +- **Flexible In Your Cloud**: Docq is deployed in your cloud account similar to the _Secure Cloud Deployment_ option above. But it's configured to use a serverless LLM service provided by one of the major cloud providers such as Azure OpenAI, AWS Bedrock Claude, or GCP Vertex Gemini Pro. This is suitable for organisations with less sensitive data. -- **Secure Cloud Deployment**: For businesses that prefer to keep all their organisational data private, Docq can be deployed and configured to operate within an organisation's cloud account (i.e. network boundary) in order to guarantee that **no data is sent to any 3rd-party** intentionally or accidentally. -- **Flexible Cloud Deployment**: For businesses having less stringent requirements on their data and 3rd-party access, Docq can be - - deployed in any major cloud infrastructure provider; - - and configured to use any major 3rd-party AI/LLMs vendors via their APIs. +- **SaaS Dedicated** - Your own dedicated instance of Docq hosted and operated by us in a Azure account dedicated to you. Your instance doesn't share any infrastructure with other organisations. -Plus +- **SaaS Multi-tenant Standard** aka [app.docq.ai](https://app.docq.ai) - An instance of Docq hosted and operated in our cloud provider account (Azure UK South region). Each organisations data is isolated but you are sharing a single instance of Docq and LLMs. This is the fastest way to get started with Docq. Head over to [https://docq.ai/#plans](https://docq.ai/#plans) to subscribe. -- **SaaS Multi-tenant** - An instance of Docq hosted and operated in our cloud provider account (Azure UK South region). Each organisations data is isolated but you are sharing a single instance of Docq and LLMs. This is the fast way to get going. Head over to [https://docq.ai/#plans](https://docq.ai/#plans) to subscribe. -- **SaaS Dedicated** - Your own dedicated instance of Docq hosted and operated in our cloud account. You don't share any infrastructure with other organisations. -- **On-Premise Deployment**: It is another option and we are happy to support you with a bespoke setup. - -Docq has been designed from ground up for the secure cloud and on-prem deployment scenarios with strict data security and privacy requirement in mind. However it should not stop all other businesses from adopting Docq with the flexible deployment setup if it suits their requirements. +Docq has been designed from ground up to be able to cater for the scenarios with the strictest data privacy and security requirements. This will require one of the isolated deployment options. If you are working with less sensitive data you can leverage one of the other deployment options to lower setup and operational costs. ![Docq network overview](../assets/Docq_network_overview.png) -The diagram above demonstrates the possibilities of have different deployment setup for: +The diagram above visualises some of the different deployment configurations. - **Data ingestion**: across-boundary _SaaS data connectors_ versus within-boundary _data storage locations_ - **LLMs utilisation**: across-boundary models via _3rd-party APIs_ versus within-boundary _vendor-hosted & self-hosted_ models @@ -36,8 +30,12 @@ If your business have clear data security and privacy guidelines, we recommend o Do consult your chosen cloud vendor for compliance. +We have infrastructure automation for Azure. We'll develop support for the others based on demand. + ### Flexible Cloud Deployment As an OSS product, Docq comes with source code that anybody could download and run anywhere in the world. Within each release, Docq offers Docker container images to be dropped into any container runtime. The choice is yours and it usually depends on your existing IT infrastructure. -The key configuration difference between _Secure_ and _Flexible_ deployments is **whether to allow 3rd-party LLMs access from Docq**, via these vendors' APIs. In a flexible deployment, it is up to you to decide the behaviour which may have implications on the compliance of your organisation's data security and privacy requirements. +The one requirement is to be able to mount a persistent drive into the container. + +The key configuration difference between _Isolated_ and _Flexible_ deployments is **whether to allow Docq to use serverless LLMs hosted by cloud vendors or LLM providers them self**. In a flexible deployment, it is up to you to decide the behaviour which may have implications on the compliance of your organisation's data security and privacy requirements. \ No newline at end of file diff --git a/docs/overview/design-principles.md b/docs/overview/design-principles.md index 5c10d240..4a43c471 100644 --- a/docs/overview/design-principles.md +++ b/docs/overview/design-principles.md @@ -27,4 +27,4 @@ All of these are reflected in Docq's user interface and user experience design. ### Commercially-friendly Licensing & Support -Last but not least, we release Docq under a licensing structure where any business could use it as-is free-of-charge. For situations where a business wants to have a commercial agreement with both licensing and support for Docq, we can make the arrangement accordingly. +Last but not least, we release Docq under a licensing structure where any business could use it as-is free-of-charge. For situations where a business wants to have a commercial agreement with both licensing and support for Docq, we can make the arrangements accordingly. From 5f3b51fb4ed3d35078da15a888a0402f80341f45 Mon Sep 17 00:00:00 2001 From: Janaka Abeywardhana Date: Fri, 15 Mar 2024 21:28:26 +0000 Subject: [PATCH 2/4] build(docs): update Material for MKDocs to latest --- poetry.lock | 23 +++++++++++------------ pyproject.toml | 2 +- 2 files changed, 12 insertions(+), 13 deletions(-) diff --git a/poetry.lock b/poetry.lock index 56734ba9..aa761bbc 100644 --- a/poetry.lock +++ b/poetry.lock @@ -1,4 +1,4 @@ -# This file is automatically @generated by Poetry 1.7.1 and should not be changed by hand. +# This file is automatically @generated by Poetry 1.6.1 and should not be changed by hand. [[package]] name = "accelerate" @@ -1331,12 +1331,12 @@ files = [ google-auth = ">=2.14.1,<3.0.dev0" googleapis-common-protos = ">=1.56.2,<2.0.dev0" grpcio = [ - {version = ">=1.49.1,<2.0dev", optional = true, markers = "python_version >= \"3.11\" and extra == \"grpc\""}, {version = ">=1.33.2,<2.0dev", optional = true, markers = "python_version < \"3.11\" and extra == \"grpc\""}, + {version = ">=1.49.1,<2.0dev", optional = true, markers = "python_version >= \"3.11\" and extra == \"grpc\""}, ] grpcio-status = [ - {version = ">=1.49.1,<2.0.dev0", optional = true, markers = "python_version >= \"3.11\" and extra == \"grpc\""}, {version = ">=1.33.2,<2.0.dev0", optional = true, markers = "python_version < \"3.11\" and extra == \"grpc\""}, + {version = ">=1.49.1,<2.0.dev0", optional = true, markers = "python_version >= \"3.11\" and extra == \"grpc\""}, ] protobuf = ">=3.19.5,<3.20.0 || >3.20.0,<3.20.1 || >3.20.1,<4.21.0 || >4.21.0,<4.21.1 || >4.21.1,<4.21.2 || >4.21.2,<4.21.3 || >4.21.3,<4.21.4 || >4.21.4,<4.21.5 || >4.21.5,<5.0.0.dev0" requests = ">=2.18.0,<3.0.0.dev0" @@ -2523,13 +2523,13 @@ mkdocs = ">=1.0.3,<2.0.0" [[package]] name = "mkdocs-material" -version = "9.4.14" +version = "9.5.13" description = "Documentation that simply works" optional = false python-versions = ">=3.8" files = [ - {file = "mkdocs_material-9.4.14-py3-none-any.whl", hash = "sha256:dbc78a4fea97b74319a6aa9a2f0be575a6028be6958f813ba367188f7b8428f6"}, - {file = "mkdocs_material-9.4.14.tar.gz", hash = "sha256:a511d3ff48fa8718b033e7e37d17abd9cc1de0fdf0244a625ca2ae2387e2416d"}, + {file = "mkdocs_material-9.5.13-py3-none-any.whl", hash = "sha256:5cbe17fee4e3b4980c8420a04cc762d8dc052ef1e10532abd4fce88e5ea9ce6a"}, + {file = "mkdocs_material-9.5.13.tar.gz", hash = "sha256:d8e4caae576312a88fd2609b81cf43d233cdbe36860d67a68702b018b425bd87"}, ] [package.dependencies] @@ -2537,7 +2537,7 @@ babel = ">=2.10,<3.0" colorama = ">=0.4,<1.0" jinja2 = ">=3.0,<4.0" markdown = ">=3.2,<4.0" -mkdocs = ">=1.5.3,<2.0" +mkdocs = ">=1.5.3,<1.6.0" mkdocs-material-extensions = ">=1.3,<2.0" paginate = ">=0.5,<1.0" pygments = ">=2.16,<3.0" @@ -2546,8 +2546,8 @@ regex = ">=2022.4" requests = ">=2.26,<3.0" [package.extras] -git = ["mkdocs-git-committers-plugin-2 (>=1.1,<2.0)", "mkdocs-git-revision-date-localized-plugin (>=1.2,<2.0)"] -imaging = ["cairosvg (>=2.6,<3.0)", "pillow (>=9.4,<10.0)"] +git = ["mkdocs-git-committers-plugin-2 (>=1.1,<2.0)", "mkdocs-git-revision-date-localized-plugin (>=1.2.4,<2.0)"] +imaging = ["cairosvg (>=2.6,<3.0)", "pillow (>=10.2,<11.0)"] recommended = ["mkdocs-minify-plugin (>=0.7,<1.0)", "mkdocs-redirects (>=1.2,<2.0)", "mkdocs-rss-plugin (>=1.6,<2.0)"] [[package]] @@ -3798,8 +3798,8 @@ files = [ [package.dependencies] numpy = [ - {version = ">=1.23.2,<2", markers = "python_version == \"3.11\""}, {version = ">=1.22.4,<2", markers = "python_version < \"3.11\""}, + {version = ">=1.23.2,<2", markers = "python_version == \"3.11\""}, ] python-dateutil = ">=2.8.2" pytz = ">=2020.1" @@ -4794,7 +4794,6 @@ files = [ {file = "PyYAML-6.0.1-cp311-cp311-win_amd64.whl", hash = "sha256:bf07ee2fef7014951eeb99f56f39c9bb4af143d8aa3c21b1677805985307da34"}, {file = "PyYAML-6.0.1-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:855fb52b0dc35af121542a76b9a84f8d1cd886ea97c84703eaa6d88e37a2ad28"}, {file = "PyYAML-6.0.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:40df9b996c2b73138957fe23a16a4f0ba614f4c0efce1e9406a184b6d07fa3a9"}, - {file = "PyYAML-6.0.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a08c6f0fe150303c1c6b71ebcd7213c2858041a7e01975da3a99aed1e7a378ef"}, {file = "PyYAML-6.0.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:6c22bec3fbe2524cde73d7ada88f6566758a8f7227bfbf93a408a9d86bcc12a0"}, {file = "PyYAML-6.0.1-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:8d4e9c88387b0f5c7d5f281e55304de64cf7f9c0021a3525bd3b1c542da3b0e4"}, {file = "PyYAML-6.0.1-cp312-cp312-win32.whl", hash = "sha256:d483d2cdf104e7c9fa60c544d92981f12ad66a457afae824d146093b8c294c54"}, @@ -6899,4 +6898,4 @@ testing = ["big-O", "jaraco.functools", "jaraco.itertools", "more-itertools", "p [metadata] lock-version = "2.0" python-versions = ">=3.10,<3.12" -content-hash = "10b1d6dc0d1a45cfab95f5d7e215a105ef0df6f57381c9777e185d2ee1a6a5d8" +content-hash = "e2e2f0d81356d57e94d36e3acde23d51b7186d55916c222965d635df3d3ba2b2" diff --git a/pyproject.toml b/pyproject.toml index 32355490..643f0207 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -48,7 +48,7 @@ google-api-python-client = "^2.104.0" google-auth-httplib2 = "^0.1.1" microsoftgraph-python = "^1.1.6" pydantic = "^2.5.2" -mkdocs-material = "^9.4.14" +mkdocs-material = "^9.5.13" pyautogen = "^0.2.2" termcolor = "^2.4.0" google-cloud-aiplatform = "^1.38.0" From 09b1a6657c1c884e3c2c26a0e2b6e9ff14708be3 Mon Sep 17 00:00:00 2001 From: Janaka Abeywardhana Date: Fri, 15 Mar 2024 21:28:51 +0000 Subject: [PATCH 3/4] docs: update styles to match purple theme on marketing site --- docs/stylesheets/extra.css | 15 +++++++++++++-- mkdocs.yaml | 22 +++++++++++++++++++--- 2 files changed, 32 insertions(+), 5 deletions(-) diff --git a/docs/stylesheets/extra.css b/docs/stylesheets/extra.css index 159be19c..34fa8a02 100644 --- a/docs/stylesheets/extra.css +++ b/docs/stylesheets/extra.css @@ -1,8 +1,12 @@ +:root { + --md-typeset-a-color: #542CF2; +} + [data-md-color-scheme="docq"] { --md-primary-fg-color: #202020; --md-primary-fg-color--light: #373737; --md-primary-fg-color--dark: #373737; - --md-typeset-a-color: #542CF2; + //--md-typeset-a-color: #542CF2; //--md-typeset-a-color: #4cae4f; } @@ -12,6 +16,7 @@ --md-primary-fg-color--dark: #373737; --md-typeset-a-color: #542CF2; //--md-typeset-a-color: #71c174; + --md-hue: 240; } .mkapi-node code.mkapi-item-name, code.mkapi-object-parenthesis, code.mkapi-object-signature, span.mkapi-item-type, span.mkapi-item-dash, mkapi-item-description { @@ -20,7 +25,7 @@ } .mkapi-node code.mkapi-item-name { - color: var(--md-typeset-a-color); + //color: var(--md-typeset-a-color); } .mkapi-node ul.mkapi-items { @@ -40,4 +45,10 @@ .mkapi-section-name { color: var(--md-default-fg-color); +} + +.md-header { + background-color: #542CF2; + background-image: linear-gradient(6deg, #784fed 40%, rgba(84,44,242,0.008) 80%); + } \ No newline at end of file diff --git a/mkdocs.yaml b/mkdocs.yaml index ee99ef89..5ddb928d 100644 --- a/mkdocs.yaml +++ b/mkdocs.yaml @@ -1,8 +1,8 @@ -site_name: Docq - Private & Secure ChatGPT AI Knowledge Insight +site_name: Your private ChatGPT alternative. site_url: https://docqai.github.io/docq site_description: Docq offers self-hosted, private and secure ChatGPT on your private data with minimal onboarding and operational effort. site_author: Docq Contributors -copyright: Copyright © 2023 Docq Contributors +copyright: Copyright © 2023 - 2024 Docq Contributors site_dir: html repo_name: docqai/docq repo_url: https://github.com/docqai/docq @@ -34,6 +34,21 @@ extra: analytics: provider: google property: G-SML7XH761K + social: + - icon: fontawesome/brands/x-twitter + link: https://twitter.com/docqai + name: docqai + - icon: fontawesome/brands/slack + link: https://join.slack.com/t/docqai/shared_invite/zt-27p17lu6v-6KLJxSmt61vfNqCavSE73A + name: Join Slack + - icon: fontawesome/brands/linkedin + link: https://www.linkedin.com/company/docqai + name: linkedin.com/company/docqai + - icon: fontawesome/brands/medium + link: https://medium.com/@docqai + name: Blog + - icon: fontawesome/regular/envelope + link: mailto: extra_css: - stylesheets/extra.css @@ -43,7 +58,7 @@ theme: logo: assets/logo.jpg favicon: assets/favicon.png language: en - palette: + palette: - media: "(prefers-color-scheme: light)" scheme: docq accent: purple @@ -51,6 +66,7 @@ theme: icon: material/lightbulb-outline name: Switch to light mode - media: "(prefers-color-scheme: dark)" + primary: custom scheme: slate accent: purple toggle: From 85015ade8726f81967279f155deb6f6e057f1979 Mon Sep 17 00:00:00 2001 From: Janaka Abeywardhana Date: Sat, 16 Mar 2024 00:40:11 +0000 Subject: [PATCH 4/4] few more tweaks --- docs/stylesheets/extra.css | 2 +- mkdocs.yaml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/stylesheets/extra.css b/docs/stylesheets/extra.css index 34fa8a02..98732e22 100644 --- a/docs/stylesheets/extra.css +++ b/docs/stylesheets/extra.css @@ -16,7 +16,7 @@ --md-primary-fg-color--dark: #373737; --md-typeset-a-color: #542CF2; //--md-typeset-a-color: #71c174; - --md-hue: 240; + //--md-hue: 140; } .mkapi-node code.mkapi-item-name, code.mkapi-object-parenthesis, code.mkapi-object-signature, span.mkapi-item-type, span.mkapi-item-dash, mkapi-item-description { diff --git a/mkdocs.yaml b/mkdocs.yaml index 5ddb928d..eb4f4b48 100644 --- a/mkdocs.yaml +++ b/mkdocs.yaml @@ -83,7 +83,7 @@ theme: - navigation.sections icon: - repo: fontawesome/brands/github-alt + repo: fontawesome/brands/github admonition: note: fontawesome/solid/note-sticky abstract: fontawesome/solid/book