Chore/update kustomizations based on new version of wren-ai-service (#…

…514)

* fix based on default image versions

* remove sqlite

* update kustomizations to the latest version of ai-service, without custom llms

* update instructions

* fix typo

deployment/kustomizations/README.md

@@ -23,20 +23,20 @@ git clone https://github.com/Canner/WrenAI.git
 cd WrenAI
 
 # Inflate the manifest with kustomization
-kubectl kustomize deployment/kustomizations --enable-helm > deployment/kustomizations/wrenai.kustimized.yaml
+kubectl kustomize deployment/kustomizations --enable-helm > deployment/kustomizations/wrenai.kustomized.yaml
 
 # Create namespace
 kubectl create namespace wren
 
 # !!!!!!!!!!!!
 # MODIFY secret-wren_example.yaml manifest file FIRST
-# OPENAI_API_KEY is REQUIRED: without a valid key the wren-ai-service-deployment pod will not start
+# LLM_OPENAI_API_KEY and EMBEDDER_OPENAI_API_KEY are REQUIRED: without a valid key the wren-ai-service-deployment pod will not start
 # You must update PG_URL, otherwise wren-ui will not work
 #vi deployment/kustomizations/examples/secret-wren_example.yaml
-kubectl apply -f deployment/kustomizations/examples/secret-wren_example.yaml
+kubectl apply -f deployment/kustomizations/examples/secret-wren_example.yaml -n wren
 
 # Deploy the app:
-kubectl apply -f deployment/kustomizations/wrenai.kustimized.yaml
+kubectl apply -f deployment/kustomizations/wrenai.kustomized.yaml
 
 kubectl get pods -n wren
 ```
@@ -52,16 +52,15 @@ kubectl get pods -n wren
 - `deployment/kustomizations/patches` folder is empty, feel free to add your own patches & overlays there.
 
 #### Wren-UI Database
-Starting with wren-ui version 0.6.0 by default the postgres database is used for wren-ui in this kuberenetes kustomization and will be installed in the same namespace as wren-ai. If you are using older version, you may use only `sqlite`.
-- `sqlite`: If you are planing to use sqlite as your database instead of postgres, you will need to uncomment `wren-sqlite` in the SQLITE_FILE, volumeMounts & PVC sections of the `deployment/kustomizations/base/deploy-wren-ui.yaml` manifest. And adjust the ConfigMap file `deployment/kustomizations/base/cm.yaml` to match your database: `DB_TYPE = "sqlite"`, comment out postgres helm chart section in the `deployment/kustomizations/kustomization.yaml` file and comment out the `PG_URL` in the Secret manifest `deployment/kustomizations/examples/secret-wren_example.yaml`.
-- `postgres` (default): Database that will be installed in the same namespace as wren-ai. You *must* update `PG_URL` in the Secret manifest `deployment/kustomizations/examples/secret-wren_example.yaml`.
+Starting with wren-ui version 0.6.0 by default the postgres database is used for wren-ui in this kuberenetes kustomization and will be installed in the same namespace as wren-ai.
+- `postgres`: Database that will be installed in the same namespace as wren-ai. You *must* update `PG_URL` in the Secret manifest `deployment/kustomizations/examples/secret-wren_example.yaml`.
 
 Example: `PG_URL: "postgres://postgres:postgres@wrenai-postgresql:5432/admin_ui"`
 - `postgres://`        This is the protocol. It tells the system that you’re connecting to a PostgreSQL database.
 - `postgres:postgres`  These are the username(first) and password(second) for the database respectively, separated by a colon. In this case, both the username and password are “postgres”.
-- `@wrenai-postgresql`   This is the hostname of the database server. "wren-postgresql" means the database server is running in a Kubernetes cluster and it is named "wren-postgresql" in the *same* namespace. If you are using another namespace you must provide the full hostname, example: `wren-postgresql.wrenai.svc.cluster.local`, "wrenai" is the namespace name, "svc.cluster.local" is the default domain name for Kubernetes services no need to change it.
+- `@wren-postgresql`   This is the hostname of the database server. "wren-postgresql" means the database server is running in a Kubernetes cluster and it is named "wren-postgresql" in the *same* namespace. If you are using another namespace you must provide the full hostname, example: `wren-postgresql.wrenai.svc.cluster.local`, "wrenai" is the namespace name, "svc.cluster.local" is the default domain name for Kubernetes services no need to change it.
 - `:5432`              This is the port number. PostgreSQL servers listen on port 5432 by default.
-- `/admin_ui`          This is the name of the database you’re connecting to. In this case, the database name is `admin_ui`. It can be found in the helm values file in the auth.database parameter `deployment/kustomizations/helm-values_postgresql_14.yaml`
+- `/admin_ui`          This is the name of the database you’re connecting to. In this case, the database name is `admin_ui`. It can be found in the helm values file in the auth.database parameter `deployment/kustomizations/helm-values_postgresql_15.yaml`
 
 # Minikube
 Prepare your k8s environment. Then use the `Steps to deploy` section to deploy Wren AI app into your k8s.

deployment/kustomizations/base/cm.yaml

@@ -18,9 +18,15 @@ data:
   WREN_AI_SERVICE_VERSION: "0.7.3"
   WREN_UI_VERSION: "0.10.1"
 
-  # OpenAI
+  # LLM and Embedder Configurations
+  LLM_PROVIDER: "openai_llm"
+  LLM_OPENAI_API_BASE: "https://api.openai.com/v1"
   GENERATION_MODEL: "gpt-3.5-turbo"
 
+  EMBEDDER_PROVIDER: "openai_embedder"
+  EMBEDDER_OPENAI_API_BASE: "https://api.openai.com/v1"
+
+
   # Telemetry
   POSTHOG_HOST: "https://app.posthog.com"
   TELEMETRY_ENABLED: "false"
@@ -30,11 +36,9 @@ data:
   WREN_AI_ENDPOINT: "http://wren-ai-service-svc:5555"
   #WREN_AI_ENDPOINT: "http://wren-ai-service-svc.ai-system.svc.cluster.local:5555"
 
-  # "sqlite", or "pg" for postgres as UI application database
+  # "pg" for postgres as UI application database
   WREN_UI_DB_TYPE: pg
 
-  ### if DB_TYPE = "sqlite" also make sure to uncomment the "wren-sqlite" in the SQLITE_FILE, volumeMounts & PVC sections of the deployment/kustomizations/base/deploy-wren-ui.yaml
-  #SQLITE_FILE: /app/data/db.sqlite3
   #For bootstrap
   WREN_ENGINE_DATA_PATH: "/app/data"
 

deployment/kustomizations/base/deploy-wren-ai-service.yaml

@@ -21,21 +21,36 @@ spec:
               configMapKeyRef:
                 name: wren-config
                 key: WREN_AI_SERVICE_PORT
-          - name: OPENAI_API_KEY
+          - name: LLM_PROVIDER
+            valueFrom:
+              configMapKeyRef:
+                name: wren-config
+                key: LLM_PROVIDER
+          - name: LLM_OPENAI_API_KEY
             valueFrom:
               secretKeyRef:
                 name: wrenai-secrets
-                key: OPENAI_API_KEY
-          - name: LLM_PROVIDER
+                key: LLM_OPENAI_API_KEY
+          - name: LLM_OPENAI_API_BASE
+            valueFrom: 
+              configMapKeyRef:
+                name: wren-config
+                key: LLM_OPENAI_API_BASE
+          - name: EMBEDDER_PROVIDER
+            valueFrom:
+              configMapKeyRef:
+                name: wren-config
+                key: EMBEDDER_PROVIDER
+          - name: EMBEDDER_OPENAI_API_KEY
             valueFrom:
               secretKeyRef:
                 name: wrenai-secrets
-                key: LLM_PROVIDER
-          - name: OPENAI_API_BASE
+                key: EMBEDDER_OPENAI_API_KEY
+          - name: EMBEDDER_OPENAI_API_BASE
             valueFrom: 
               configMapKeyRef:
                 name: wren-config
-                key: OPENAI_API_BASE
+                key: EMBEDDER_OPENAI_API_BASE
           - name: GENERATION_MODEL
             valueFrom: 
               configMapKeyRef:

deployment/kustomizations/base/deploy-wren-ui.yaml

@@ -12,12 +12,6 @@ spec:
       labels:
         app: wren-ui
     spec:
-      #if you are using sqlite, then volume is required, please uncomment next, SQLITE_FILE and the volumeMounts sections
-      #if you are using postgres, then volume is not required
-      #volumes:
-      #  - name: wren-sqlite
-      #    persistentVolumeClaim:
-      #      claimName: wren-sqlite-pvc
       containers:
 
         - name: wren-ui
@@ -28,12 +22,6 @@ spec:
                 configMapKeyRef:
                   name: wren-config
                   key: WREN_UI_DB_TYPE
-          # Uncomment if you are intending to use sqlite:
-          #  - name: SQLITE_FILE
-          #    valueFrom:
-          #      configMapKeyRef:
-          #        name: wren-config
-          #        key: SQLITE_FILE
             - name: WREN_ENGINE_ENDPOINT
               valueFrom:
                 configMapKeyRef:
@@ -129,8 +117,3 @@ spec:
                   key: IBIS_SERVER_ENDPOINT
           ports:
             - containerPort: 3000
-          #if you are using sqlite, then volume is required, please uncomment next and the persistentVolumeClaim sections
-          #if you are using postgres, then volume is not required
-          #volumeMounts:
-          #  - name: wren-sqlite
-          #    mountPath: /app/data

deployment/kustomizations/examples/secret-wren_example.yaml

@@ -5,10 +5,9 @@ metadata:
   name: wrenai-secrets
 type: Opaque
 data:
-  # OPENAI_API_KEY is REQUIRED: without a valid key the wren-ai-service-deployment pod will not start
-  OPENAI_API_KEY: UkVRVUlSRUQ6IHNrLXByb2otYWxsLWFjY2Vzcy1wbGFjZWhvbGRlci00LXdyZW4tYWktc2VydmljZS1kZXBsb3ltZW50
-  OPENAI_API_BASE: aHR0cHM6Ly9hcGkub3BlbmFpLmNvbS92MQ==
-  LLM_PROVIDER: b3BlbmFp
+  # LLM_OPENAI_API_KEY and EMBEDDER_OPENAI_API_KEY are REQUIRED: without a valid key the wren-ai-service-deployment pod will not start
+  LLM_OPENAI_API_KEY: UkVRVUlSRUQ6IHNrLXByb2otYWxsLWFjY2Vzcy1wbGFjZWhvbGRlci00LXdyZW4tYWktc2VydmljZS1kZXBsb3ltZW50
+  EMBEDDER_OPENAI_API_KEY: UkVRVUlSRUQ6IHNrLXByb2otYWxsLWFjY2Vzcy1wbGFjZWhvbGRlci00LXdyZW4tYWktc2VydmljZS1kZXBsb3ltZW50  
   PG_USERNAME: cG9zdGdyZXM=
 
   #Azure openai env
@@ -20,16 +19,14 @@ data:
   AZURE_EMBED_KEY: bi9h
   AZURE_EMBED_VERSION: bi9h
 
-  ### if in ConfigMap DB_TYPE = "sqlite" you must comment out the PG_URL string here
-  ### if in ConfigMap DB_TYPE = "postgres" you must provide PG_URL string in this Secret manifest file below to connect to postgres
   ### postgres://        This is the protocol. It tells the system that you’re connecting to a PostgreSQL database.
   ### postgres:postgres  These are the username and password for the database, separated by a colon. In this case, both the username and password are “postgres”.
-  ### @wrenai-postgresql   This is the hostname of the database server. "wrenai-postgresql" means the database server is running in a Kubernetes cluster and it is named "wren-postgresql" in the *same* namespace. If you are using another namespace you must provide the full hostname, example: `wren-postgresql.wrenai.svc.cluster.local`, "wrenai" is the namespace name, "svc.cluster.local" is the default domain name for Kubernetes services no need to change it.
+  ### @wren-postgresql   This is the hostname of the database server. "wren-postgresql" means the database server is running in a Kubernetes cluster and it is named "wren-postgresql" in the *same* namespace. If you are using another namespace you must provide the full hostname, example: `wren-postgresql.wrenai.svc.cluster.local`, "wrenai" is the namespace name, "svc.cluster.local" is the default domain name for Kubernetes services no need to change it.
   ### :5432              This is the port number. PostgreSQL servers listen on port 5432 by default.
   ### /admin_ui          This is the name of the database you’re connecting to. In this case, the database name is “admin_ui”. It can be found in the helm values file in the auth.database parameter (deployment/kustomizations/helm-values_postgresql_14.yaml)
-  ### PG_URL: "postgres://postgres:postgres@wrenai-postgresql:5432/admin_ui"
+  ### PG_URL: "postgres://postgres:postgres@wren-postgresql:5432/admin_ui"
   #Fix
-  PG_URL: cG9zdGdyZXM6Ly9wb3N0Z3Jlczpwb3N0Z3Jlc0B3cmVuYWktcG9zdGdyZXNxbDo1NDMyL2FkbWluX3Vp
+  PG_URL: cG9zdGdyZXM6Ly9wb3N0Z3Jlczpwb3N0Z3Jlc0B3cmVuLXBvc3RncmVzcWw6NTQzMi9hZG1pbl91aQo=
 
   POSTHOG_API_KEY: cGhjX2tleS1wbGFjZWhvbGRlcg==
   POSTHOG_HOST: aHR0cHM6Ly9hcHAucG9zdGhvZy5jb20=

deployment/kustomizations/kustomization.yaml

@@ -16,14 +16,14 @@ namespace: wren
 helmCharts:
 
 ### Uncomment if you are planing to use postgresql
-#  - name: postgresql
-#    repo: https://charts.bitnami.com/bitnami
-#    version: 14.3.3
-#    releaseName: wren-postgresql
-#    valuesFile: helm-values_postgresql_14.yaml
-#    includeCRDs: true
-#    # the Same Namespace
-#    namespace: wren
+  - name: postgresql
+    repo: https://repo.vmware.com/bitnami-files
+    version: 15.5.5
+    releaseName: wren-postgresql
+    valuesFile: helm-values_postgresql_15.yaml
+    includeCRDs: true
+    # the Same Namespace
+    namespace: wren
 
   - name: qdrant
     repo: https://qdrant.github.io/qdrant-helm
@@ -39,13 +39,13 @@ images:
   - name: ghcr.io/canner/wren-bootstrap
     newTag: 0.1.4 # WREN_BOOTSTRAP_VERSION
   - name: ghcr.io/canner/wren-engine
-    newTag: 0.5.1 # WREN_ENGINE_VERSION
+    newTag: 0.7.0 # WREN_ENGINE_VERSION
   - name: ghcr.io/canner/wren-ui
-    newTag: 0.8.0 # WREN_UI_VERSION Mimumal version is 0.6.0 to use postgress for wren-ui, if you are using older version, you may use only sqlite, please adjust ConfigMap (deployment/kustomizations/base/cm.yaml), Secret (deployment/kustomizations/examples/secret-wren_example.yaml) and the deployment/kustomizations/base/deploy-wren-ui.yaml manifests.
+    newTag: 0.10.0 # WREN_UI_VERSION
   - name: ghcr.io/canner/wren-ai-service
-    newTag: 0.5.7 # WREN_AI_SERVICE_VERSION
+    newTag: 0.7.2 # WREN_AI_SERVICE_VERSION
   - name: ghcr.io/canner/wren-engine-ibis
-    newTag: 0.5.1 # IBIS_SERVER_VERSION
+    newTag: 0.7.0 # IBIS_SERVER_VERSION
 
 resources:
   - base/cm.yaml

deployment/kustomizations/patches/cm.yaml

@@ -26,12 +26,10 @@
     #fix:
     WREN_AI_ENDPOINT: "http://wren-ai-service:5555"
 
-    # "sqlite", or "pg" for postgres as application database.
+    # "pg" for postgres as application database.
     #fix
     DB_TYPE: pg
 
-    ### if DB_TYPE = "sqlite" also make sure to uncomment the "wren-sqlite" in the volumeMounts & PVC sections of the deployment/kustomizations/base/deploy-wren-ui.yaml
-    #SQLITE_FILE: /app/data/db.sqlite3
     DATA_PATH: "/app/data"
 
     ### if DB_TYPE = "postgres" you must provide PG_URL string in the *Secret* manifest file (deployment/kustomizations/examples/secret-wren_example.yaml) to connect to postgres