Do not enable flyte agent by default in flyte-core helm chart (#3948)

* Do not enable flyte agent by default in flyte-core helm chart Signed-off-by: Eduardo Apolinario <[email protected]> * make helm Signed-off-by: Eduardo Apolinario <[email protected]> * Move definition of agents stanza to comment Signed-off-by: Eduardo Apolinario <[email protected]> --------- Signed-off-by: Eduardo Apolinario <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]>
flyteorg · Aug 11, 2023 · 063a77b · 063a77b
1 parent dfb56f4
commit 063a77b
Show file tree

Hide file tree

Showing 7 changed files with 25 additions and 48 deletions.
diff --git a/charts/flyte-core/README.md b/charts/flyte-core/README.md
@@ -91,10 +91,10 @@ helm install gateway bitnami/contour -n flyte
 | configmap.core.propeller | object | `{"downstream-eval-duration":"30s","enable-admin-launcher":true,"leader-election":{"enabled":true,"lease-duration":"15s","lock-config-map":{"name":"propeller-leader","namespace":"flyte"},"renew-deadline":"10s","retry-period":"2s"},"limit-namespace":"all","max-workflow-retries":30,"metadata-prefix":"metadata/propeller","metrics-prefix":"flyte","prof-port":10254,"queue":{"batch-size":-1,"batching-interval":"2s","queue":{"base-delay":"5s","capacity":1000,"max-delay":"120s","rate":100,"type":"maxof"},"sub-queue":{"capacity":100,"rate":10,"type":"bucket"},"type":"batch"},"rawoutput-prefix":"s3://my-s3-bucket/","workers":4,"workflow-reeval-duration":"30s"}` | follows the structure specified [here](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/config). |
 | configmap.datacatalogServer | object | `{"application":{"grpcPort":8089,"grpcServerReflection":true,"httpPort":8080},"datacatalog":{"heartbeat-grace-period-multiplier":3,"max-reservation-heartbeat":"30s","metrics-scope":"datacatalog","profiler-port":10254,"storage-prefix":"metadata/datacatalog"}}` | Datacatalog server config |
 | configmap.domain | object | `{"domains":[{"id":"development","name":"development"},{"id":"staging","name":"staging"},{"id":"production","name":"production"}]}` | Domains configuration for Flyte projects. This enables the specified number of domains across all projects in Flyte. |
-| configmap.enabled_plugins.tasks | object | `{"task-plugins":{"default-for-task-types":{"bigquery_query_job_task":"agent-service","container":"container","container_array":"k8s-array","sidecar":"sidecar"},"enabled-plugins":["container","sidecar","k8s-array","agent-service"]}}` | Tasks specific configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#GetConfig) |
-| configmap.enabled_plugins.tasks.task-plugins | object | `{"default-for-task-types":{"bigquery_query_job_task":"agent-service","container":"container","container_array":"k8s-array","sidecar":"sidecar"},"enabled-plugins":["container","sidecar","k8s-array","agent-service"]}` | Plugins configuration, [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#TaskPluginConfig) |
-| configmap.enabled_plugins.tasks.task-plugins.enabled-plugins | list | `["container","sidecar","k8s-array","agent-service"]` | [Enabled Plugins](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/config#Config). Enable sagemaker*, athena if you install the backend plugins |
-| configmap.k8s | object | `{"plugins":{"agent-service":{"defaultGrpcEndpoint":"flyteagent.flyte.svc.cluster.local:8000","supportedTaskTypes":["bigquery_query_job_task"]},"k8s":{"default-cpus":"100m","default-env-vars":[],"default-memory":"100Mi"}}}` | Kubernetes specific Flyte configuration |
+| configmap.enabled_plugins.tasks | object | `{"task-plugins":{"default-for-task-types":{"container":"container","container_array":"k8s-array","sidecar":"sidecar"},"enabled-plugins":["container","sidecar","k8s-array"]}}` | Tasks specific configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#GetConfig) |
+| configmap.enabled_plugins.tasks.task-plugins | object | `{"default-for-task-types":{"container":"container","container_array":"k8s-array","sidecar":"sidecar"},"enabled-plugins":["container","sidecar","k8s-array"]}` | Plugins configuration, [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#TaskPluginConfig) |
+| configmap.enabled_plugins.tasks.task-plugins.enabled-plugins | list | `["container","sidecar","k8s-array"]` | [Enabled Plugins](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/config#Config). Enable sagemaker*, athena if you install the backend plugins |
+| configmap.k8s | object | `{"plugins":{"k8s":{"default-cpus":"100m","default-env-vars":[],"default-memory":"100Mi"}}}` | Kubernetes specific Flyte configuration |
 | configmap.k8s.plugins.k8s | object | `{"default-cpus":"100m","default-env-vars":[],"default-memory":"100Mi"}` | Configuration section for all K8s specific plugins [Configuration structure](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/pluginmachinery/flytek8s/config) |
 | configmap.remoteData.remoteData.region | string | `"us-east-1"` |  |
 | configmap.remoteData.remoteData.scheme | string | `"local"` |  |
@@ -172,7 +172,7 @@ helm install gateway bitnami/contour -n flyte
 | flyteagent.additionalVolumes | list | `[]` | Appends additional volumes to the deployment spec. May include template values. |
 | flyteagent.affinity | object | `{}` | affinity for flyteagent deployment |
 | flyteagent.configPath | string | `"/etc/flyteagent/config/*.yaml"` | Default regex string for searching configuration files |
-| flyteagent.enabled | bool | `true` |  |
+| flyteagent.enabled | bool | `false` |  |
 | flyteagent.extraArgs | object | `{}` | Appends extra command line arguments to the main command |
 | flyteagent.image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy |
 | flyteagent.image.repository | string | `"ghcr.io/flyteorg/flyteagent"` | Docker image for flyteagent deployment |

diff --git a/charts/flyte-core/values.yaml b/charts/flyte-core/values.yaml
@@ -203,7 +203,7 @@ datacatalog:
 #
 
 flyteagent:
-  enabled: true
+  enabled: false
   # -- Replicas count for flyteagent deployment
   replicaCount: 1
   image:
@@ -718,12 +718,14 @@ configmap:
           - container
           - sidecar
           - k8s-array
-          - agent-service
+          # -- Uncomment to enable agent service
+          # - agent-service
         default-for-task-types:
           container: container
           sidecar: sidecar
           container_array: k8s-array
-          bigquery_query_job_task: agent-service
+          # -- Uncomment to enable task type that uses Flyte Agent
+          # bigquery_query_job_task: agent-service
 
   # -- Kubernetes specific Flyte configuration
   k8s:
@@ -734,11 +736,11 @@ configmap:
         #  DEFAULT_ENV_VAR: VALUE
         default-cpus: 100m
         default-memory: 100Mi
-      agent-service:
-        defaultGrpcEndpoint: flyteagent.flyte.svc.cluster.local:8000
-        supportedTaskTypes:
-          - bigquery_query_job_task
-
+      # Uncomment and modify to include configuration for Flyte Agent
+      # agent-service:
+      #   defaultGrpcEndpoint: flyteagent.flyte.svc.cluster.local:8000
+      #   supportedTaskTypes:
+      #     - bigquery_query_job_task
   remoteData:
     remoteData:
       region: "us-east-1"

diff --git a/deployment/eks/flyte_aws_scheduler_helm_generated.yaml b/deployment/eks/flyte_aws_scheduler_helm_generated.yaml
@@ -482,7 +482,6 @@ data:
     tasks:
       task-plugins:
         default-for-task-types:
-          bigquery_query_job_task: agent-service
           container: container
           container_array: k8s-array
           sidecar: sidecar
@@ -492,10 +491,6 @@ data:
         - k8s-array
   k8s.yaml: | 
     plugins:
-      agent-service:
-        defaultGrpcEndpoint: flyteagent.flyte.svc.cluster.local:8000
-        supportedTaskTypes:
-        - bigquery_query_job_task
       k8s:
         default-cpus: 100m
         default-env-vars: []
@@ -1234,7 +1229,7 @@ spec:
   template:
     metadata:
       annotations:
-        configChecksum: "5aeb1eaeab35d20dc21450af4b772c377f0d5418905bf6eaf170a6e5ebaade1"
+        configChecksum: "4ba4974b5cb243e068ebba9501891f71b52aada9c279a0ba7b6e2943abcc7f6"
       labels: 
         app.kubernetes.io/name: flytepropeller
         app.kubernetes.io/instance: flyte
@@ -1316,7 +1311,7 @@ spec:
         app.kubernetes.io/name: flyte-pod-webhook
         app.kubernetes.io/version: v1.1.106
       annotations:
-        configChecksum: "5aeb1eaeab35d20dc21450af4b772c377f0d5418905bf6eaf170a6e5ebaade1"
+        configChecksum: "4ba4974b5cb243e068ebba9501891f71b52aada9c279a0ba7b6e2943abcc7f6"
     spec:
       securityContext:
         fsGroup: 65534

diff --git a/deployment/eks/flyte_helm_dataplane_generated.yaml b/deployment/eks/flyte_helm_dataplane_generated.yaml
@@ -147,7 +147,6 @@ data:
     tasks:
       task-plugins:
         default-for-task-types:
-          bigquery_query_job_task: agent-service
           container: container
           container_array: k8s-array
           sidecar: sidecar
@@ -157,10 +156,6 @@ data:
         - k8s-array
   k8s.yaml: | 
     plugins:
-      agent-service:
-        defaultGrpcEndpoint: flyteagent.flyte.svc.cluster.local:8000
-        supportedTaskTypes:
-        - bigquery_query_job_task
       k8s:
         default-cpus: 100m
         default-env-vars: []
@@ -431,7 +426,7 @@ spec:
   template:
     metadata:
       annotations:
-        configChecksum: "5aeb1eaeab35d20dc21450af4b772c377f0d5418905bf6eaf170a6e5ebaade1"
+        configChecksum: "4ba4974b5cb243e068ebba9501891f71b52aada9c279a0ba7b6e2943abcc7f6"
       labels: 
         app.kubernetes.io/name: flytepropeller
         app.kubernetes.io/instance: flyte
@@ -513,7 +508,7 @@ spec:
         app.kubernetes.io/name: flyte-pod-webhook
         app.kubernetes.io/version: v1.1.106
       annotations:
-        configChecksum: "5aeb1eaeab35d20dc21450af4b772c377f0d5418905bf6eaf170a6e5ebaade1"
+        configChecksum: "4ba4974b5cb243e068ebba9501891f71b52aada9c279a0ba7b6e2943abcc7f6"
     spec:
       securityContext:
         fsGroup: 65534

diff --git a/deployment/eks/flyte_helm_generated.yaml b/deployment/eks/flyte_helm_generated.yaml
@@ -513,7 +513,6 @@ data:
     tasks:
       task-plugins:
         default-for-task-types:
-          bigquery_query_job_task: agent-service
           container: container
           container_array: k8s-array
           sidecar: sidecar
@@ -523,10 +522,6 @@ data:
         - k8s-array
   k8s.yaml: | 
     plugins:
-      agent-service:
-        defaultGrpcEndpoint: flyteagent.flyte.svc.cluster.local:8000
-        supportedTaskTypes:
-        - bigquery_query_job_task
       k8s:
         default-cpus: 100m
         default-env-vars: []
@@ -1353,7 +1348,7 @@ spec:
   template:
     metadata:
       annotations:
-        configChecksum: "5aeb1eaeab35d20dc21450af4b772c377f0d5418905bf6eaf170a6e5ebaade1"
+        configChecksum: "4ba4974b5cb243e068ebba9501891f71b52aada9c279a0ba7b6e2943abcc7f6"
       labels: 
         app.kubernetes.io/name: flytepropeller
         app.kubernetes.io/instance: flyte
@@ -1435,7 +1430,7 @@ spec:
         app.kubernetes.io/name: flyte-pod-webhook
         app.kubernetes.io/version: v1.1.106
       annotations:
-        configChecksum: "5aeb1eaeab35d20dc21450af4b772c377f0d5418905bf6eaf170a6e5ebaade1"
+        configChecksum: "4ba4974b5cb243e068ebba9501891f71b52aada9c279a0ba7b6e2943abcc7f6"
     spec:
       securityContext:
         fsGroup: 65534

diff --git a/deployment/gcp/flyte_helm_dataplane_generated.yaml b/deployment/gcp/flyte_helm_dataplane_generated.yaml
@@ -147,7 +147,6 @@ data:
     tasks:
       task-plugins:
         default-for-task-types:
-          bigquery_query_job_task: agent-service
           container: container
           container_array: k8s-array
           sidecar: sidecar
@@ -157,10 +156,6 @@ data:
         - k8s-array
   k8s.yaml: | 
     plugins:
-      agent-service:
-        defaultGrpcEndpoint: flyteagent.flyte.svc.cluster.local:8000
-        supportedTaskTypes:
-        - bigquery_query_job_task
       k8s:
         default-cpus: 100m
         default-env-vars: []
@@ -439,7 +434,7 @@ spec:
   template:
     metadata:
       annotations:
-        configChecksum: "33e17b26b38185e0f1ea13266674081f74556c21248f035d3a5cc7dbd32ee37"
+        configChecksum: "d79d5b7a6b0475a27cdd4b85f848f0be4bfed0430ec68184d4c165acf0e8329"
       labels: 
         app.kubernetes.io/name: flytepropeller
         app.kubernetes.io/instance: flyte
@@ -520,7 +515,7 @@ spec:
         app.kubernetes.io/name: flyte-pod-webhook
         app.kubernetes.io/version: v1.1.106
       annotations:
-        configChecksum: "33e17b26b38185e0f1ea13266674081f74556c21248f035d3a5cc7dbd32ee37"
+        configChecksum: "d79d5b7a6b0475a27cdd4b85f848f0be4bfed0430ec68184d4c165acf0e8329"
     spec:
       securityContext:
         fsGroup: 65534

diff --git a/deployment/gcp/flyte_helm_generated.yaml b/deployment/gcp/flyte_helm_generated.yaml
@@ -526,7 +526,6 @@ data:
     tasks:
       task-plugins:
         default-for-task-types:
-          bigquery_query_job_task: agent-service
           container: container
           container_array: k8s-array
           sidecar: sidecar
@@ -536,10 +535,6 @@ data:
         - k8s-array
   k8s.yaml: | 
     plugins:
-      agent-service:
-        defaultGrpcEndpoint: flyteagent.flyte.svc.cluster.local:8000
-        supportedTaskTypes:
-        - bigquery_query_job_task
       k8s:
         default-cpus: 100m
         default-env-vars: []
@@ -1376,7 +1371,7 @@ spec:
   template:
     metadata:
       annotations:
-        configChecksum: "33e17b26b38185e0f1ea13266674081f74556c21248f035d3a5cc7dbd32ee37"
+        configChecksum: "d79d5b7a6b0475a27cdd4b85f848f0be4bfed0430ec68184d4c165acf0e8329"
       labels: 
         app.kubernetes.io/name: flytepropeller
         app.kubernetes.io/instance: flyte
@@ -1457,7 +1452,7 @@ spec:
         app.kubernetes.io/name: flyte-pod-webhook
         app.kubernetes.io/version: v1.1.106
       annotations:
-        configChecksum: "33e17b26b38185e0f1ea13266674081f74556c21248f035d3a5cc7dbd32ee37"
+        configChecksum: "d79d5b7a6b0475a27cdd4b85f848f0be4bfed0430ec68184d4c165acf0e8329"
     spec:
       securityContext:
         fsGroup: 65534