Skip to content

Commit

Permalink
Update Flyte components (#3408)
Browse files Browse the repository at this point in the history
* Update Flyte Components

Signed-off-by: Flyte-Bot <[email protected]>

* Copy dist files from right place.

Signed-off-by: Eduardo Apolinario <[email protected]>

* Update conf.py version

Signed-off-by: Eduardo Apolinario <[email protected]>

* Add changelog

Signed-off-by: Eduardo Apolinario <[email protected]>

---------

Signed-off-by: Flyte-Bot <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Co-authored-by: eapolinario <[email protected]>
  • Loading branch information
flyte-bot and eapolinario authored Mar 7, 2023
1 parent b7afefa commit eba986b
Show file tree
Hide file tree
Showing 25 changed files with 241 additions and 174 deletions.
67 changes: 67 additions & 0 deletions CHANGELOG/CHANGELOG-v1.4.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# Flyte 1.4 release

The main features of the 1.4 release are:
- Suport for `PodTemplate` at the task-level
- Revamped auth system in flytekit

As python 3.7 [reached](https://endoflife.date/python) EOL support in December of 2022, we dropped support for that version on this release.

## Platform

### Support for `PodTemplate` at the task-level.
Users can now define [PodTemplate](https://docs.flyte.org/en/latest/deployment/configuration/general.html#using-default-k8s-podtemplates) as part of the definition of a task. For example, note how we have access a full [V1PodSpec](https://github.com/kubernetes-client/python/blob/master/kubernetes/docs/V1PodSpec.md) as part of the task definition:

```python=
@task(
pod_template=PodTemplate(
primary_container_name="primary",
labels={"lKeyA": "lValA", "lKeyB": "lValB"},
annotations={"aKeyA": "aValA", "aKeyB": "aValB"},
pod_spec=V1PodSpec(
containers=[
V1Container(
name="primary",
image="repo/placeholderImage:0.0.0",
command="echo",
args=["wow"],
resources=V1ResourceRequirements(limits={"cpu": "999", "gpu": "999"}),
env=[V1EnvVar(name="eKeyC", value="eValC"), V1EnvVar(name="eKeyD", value="eValD")],
),
],
volumes=[V1Volume(name="volume")],
tolerations=[
V1Toleration(
key="num-gpus",
operator="Equal",
value=1,
effect="NoSchedule",
),
],
)
)
)
def t1(i: str):
...
```

We are working on more examples in our documentation. Stay tuned!

## Flytekit

As promised in https://github.com/flyteorg/flytekit/releases/tag/v1.3.0, we're backporting important changes to the 1.2.x release branch. In the past month we had 2 releases: https://github.com/flyteorg/flytekit/releases/tag/v1.2.8 and https://github.com/flyteorg/flytekit/releases/tag/v1.2.9.

Here's some of the highlights of this release. For a full changelog please visit https://github.com/flyteorg/flytekit/releases/tag/v1.4.0.

### Revamped auth system
In https://github.com/flyteorg/flytekit/pull/1458 we introduced a new OAuth2 handling system based on [client-side grpc interceptors](https://grpc.github.io/grpc/python/grpc.html#client-side-interceptor).

## New sandbox features
In this new release `flytectl demo` brings the following new features:
- Support for specifying extra configuration for Flyte
- Support for specifying extra cluster resource templates for boostrapping new namespaces
- Sandbox state (DB, buckets) is now persistent across restarts and upgrades

## Flyteconsole
- [Added domain settings for project dashboard](https://github.com/flyteorg/flyteconsole/pull/689)
- [Added support for ApprovedCondition for GateNodes](https://github.com/flyteorg/flyteconsole/pull/688)
- [Performance refactor for viewing dynamic nodes](https://github.com/flyteorg/flyteconsole/pull/680)
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ WORKDIR /flyteorg/build
COPY go.mod go.sum ./
RUN go mod download
COPY cmd cmd
COPY --from=flyteconsole /app/dist cmd/single/dist
COPY --from=flyteconsole /app/ cmd/single/dist
RUN --mount=type=cache,target=/root/.cache/go-build --mount=type=cache,target=/root/go/pkg/mod \
go build -tags console -v -o dist/flyte cmd/main.go

Expand Down
8 changes: 4 additions & 4 deletions charts/flyte-core/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ helm install gateway bitnami/contour -n flyte
| flyteadmin.extraArgs | object | `{}` | Appends extra command line arguments to the serve command |
| flyteadmin.image.pullPolicy | string | `"IfNotPresent"` | |
| flyteadmin.image.repository | string | `"cr.flyte.org/flyteorg/flyteadmin"` | Docker image for Flyteadmin deployment |
| flyteadmin.image.tag | string | `"v1.1.72"` | |
| flyteadmin.image.tag | string | `"v1.1.77"` | |
| flyteadmin.initialProjects | list | `["flytesnacks","flytetester","flyteexamples"]` | Initial projects to create |
| flyteadmin.nodeSelector | object | `{}` | nodeSelector for Flyteadmin deployment |
| flyteadmin.podAnnotations | object | `{}` | Annotations for Flyteadmin pods |
Expand All @@ -173,7 +173,7 @@ helm install gateway bitnami/contour -n flyte
| flyteconsole.ga.tracking_id | string | `"G-0QW4DJWJ20"` | |
| flyteconsole.image.pullPolicy | string | `"IfNotPresent"` | |
| flyteconsole.image.repository | string | `"cr.flyte.org/flyteorg/flyteconsole"` | Docker image for Flyteconsole deployment |
| flyteconsole.image.tag | string | `"v1.4.1"` | |
| flyteconsole.image.tag | string | `"v1.4.3"` | |
| flyteconsole.nodeSelector | object | `{}` | nodeSelector for Flyteconsole deployment |
| flyteconsole.podAnnotations | object | `{}` | Annotations for Flyteconsole pods |
| flyteconsole.priorityClassName | string | `""` | Sets priorityClassName for flyte console pod(s). |
Expand All @@ -193,7 +193,7 @@ helm install gateway bitnami/contour -n flyte
| flytepropeller.extraArgs | object | `{}` | Appends extra command line arguments to the main command |
| flytepropeller.image.pullPolicy | string | `"IfNotPresent"` | |
| flytepropeller.image.repository | string | `"cr.flyte.org/flyteorg/flytepropeller"` | Docker image for Flytepropeller deployment |
| flytepropeller.image.tag | string | `"v1.1.64"` | |
| flytepropeller.image.tag | string | `"v1.1.69"` | |
| flytepropeller.manager | bool | `false` | |
| flytepropeller.nodeSelector | object | `{}` | nodeSelector for Flytepropeller deployment |
| flytepropeller.podAnnotations | object | `{}` | Annotations for Flytepropeller pods |
Expand All @@ -213,7 +213,7 @@ helm install gateway bitnami/contour -n flyte
| flytescheduler.configPath | string | `"/etc/flyte/config/*.yaml"` | Default regex string for searching configuration files |
| flytescheduler.image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy |
| flytescheduler.image.repository | string | `"cr.flyte.org/flyteorg/flytescheduler"` | Docker image for Flytescheduler deployment |
| flytescheduler.image.tag | string | `"v1.1.72"` | Docker image tag |
| flytescheduler.image.tag | string | `"v1.1.77"` | Docker image tag |
| flytescheduler.nodeSelector | object | `{}` | nodeSelector for Flytescheduler deployment |
| flytescheduler.podAnnotations | object | `{}` | Annotations for Flytescheduler pods |
| flytescheduler.priorityClassName | string | `""` | Sets priorityClassName for flyte scheduler pod(s). |
Expand Down
8 changes: 4 additions & 4 deletions charts/flyte-core/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ flyteadmin:
image:
# -- Docker image for Flyteadmin deployment
repository: cr.flyte.org/flyteorg/flyteadmin # FLYTEADMIN_IMAGE
tag: v1.1.72 # FLYTEADMIN_TAG
tag: v1.1.77 # FLYTEADMIN_TAG
pullPolicy: IfNotPresent
# -- Additional flyteadmin container environment variables
#
Expand Down Expand Up @@ -97,7 +97,7 @@ flytescheduler:
# -- Docker image for Flytescheduler deployment
repository: cr.flyte.org/flyteorg/flytescheduler # FLYTESCHEDULER_IMAGE
# -- Docker image tag
tag: v1.1.72 # FLYTESCHEDULER_TAG
tag: v1.1.77 # FLYTESCHEDULER_TAG
# -- Docker image pull policy
pullPolicy: IfNotPresent
# -- Default resources requests and limits for Flytescheduler deployment
Expand Down Expand Up @@ -212,7 +212,7 @@ flytepropeller:
image:
# -- Docker image for Flytepropeller deployment
repository: cr.flyte.org/flyteorg/flytepropeller # FLYTEPROPELLER_IMAGE
tag: v1.1.64 # FLYTEPROPELLER_TAG
tag: v1.1.69 # FLYTEPROPELLER_TAG
pullPolicy: IfNotPresent
# -- Default resources requests and limits for Flytepropeller deployment
resources:
Expand Down Expand Up @@ -270,7 +270,7 @@ flyteconsole:
image:
# -- Docker image for Flyteconsole deployment
repository: cr.flyte.org/flyteorg/flyteconsole # FLYTECONSOLE_IMAGE
tag: v1.4.1 # FLYTECONSOLE_TAG
tag: v1.4.3 # FLYTECONSOLE_TAG
pullPolicy: IfNotPresent
# -- Default resources requests and limits for Flyteconsole deployment
resources:
Expand Down
10 changes: 5 additions & 5 deletions charts/flyte/README.md

Large diffs are not rendered by default.

8 changes: 4 additions & 4 deletions charts/flyte/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ flyte:
# -- Docker image for Flyteadmin deployment
repository: cr.flyte.org/flyteorg/flyteadmin # FLYTEADMIN_IMAGE
# -- Docker image tag
tag: v1.1.72 # FLYTEADMIN_TAG
tag: v1.1.77 # FLYTEADMIN_TAG
# -- Docker image pull policy
pullPolicy: IfNotPresent
# -- Additional flyteadmin container environment variables
Expand Down Expand Up @@ -84,7 +84,7 @@ flyte:
# -- Docker image for Flytescheduler deployment
repository: cr.flyte.org/flyteorg/flytescheduler # FLYTESCHEDULER_IMAGE
# -- Docker image tag
tag: v1.1.72 # FLYTESCHEDULER_TAG
tag: v1.1.77 # FLYTESCHEDULER_TAG
# -- Docker image pull policy
pullPolicy: IfNotPresent
# -- Default resources requests and limits for Flytescheduler deployment
Expand Down Expand Up @@ -178,7 +178,7 @@ flyte:
# -- Docker image for Flytepropeller deployment
repository: cr.flyte.org/flyteorg/flytepropeller # FLYTEPROPELLER_IMAGE
# -- Docker image tag
tag: v1.1.64 # FLYTEPROPELLER_TAG
tag: v1.1.69 # FLYTEPROPELLER_TAG
# -- Docker image pull policy
pullPolicy: IfNotPresent
# -- Default resources requests and limits for Flytepropeller deployment
Expand Down Expand Up @@ -223,7 +223,7 @@ flyte:
# -- Docker image for Flyteconsole deployment
repository: cr.flyte.org/flyteorg/flyteconsole # FLYTECONSOLE_IMAGE
# -- Docker image tag
tag: v1.4.1 # FLYTECONSOLE_TAG
tag: v1.4.3 # FLYTECONSOLE_TAG
# -- Docker image pull policy
pullPolicy: IfNotPresent
# -- Default resources requests and limits for Flyteconsole deployment
Expand Down
22 changes: 11 additions & 11 deletions deployment/eks/flyte_aws_scheduler_helm_generated.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -868,7 +868,7 @@ spec:
- /etc/flyte/config/*.yaml
- migrate
- run
image: "cr.flyte.org/flyteorg/flyteadmin:v1.1.72"
image: "cr.flyte.org/flyteorg/flyteadmin:v1.1.77"
imagePullPolicy: "IfNotPresent"
name: run-migrations
volumeMounts:
Expand All @@ -885,7 +885,7 @@ spec:
- flytesnacks
- flytetester
- flyteexamples
image: "cr.flyte.org/flyteorg/flyteadmin:v1.1.72"
image: "cr.flyte.org/flyteorg/flyteadmin:v1.1.77"
imagePullPolicy: "IfNotPresent"
name: seed-projects
volumeMounts:
Expand All @@ -899,7 +899,7 @@ spec:
- /etc/flyte/config/*.yaml
- clusterresource
- sync
image: "cr.flyte.org/flyteorg/flyteadmin:v1.1.72"
image: "cr.flyte.org/flyteorg/flyteadmin:v1.1.77"
imagePullPolicy: "IfNotPresent"
name: sync-cluster-resources
volumeMounts:
Expand All @@ -912,7 +912,7 @@ spec:
- mountPath: /etc/secrets/
name: admin-secrets
- name: generate-secrets
image: "cr.flyte.org/flyteorg/flyteadmin:v1.1.72"
image: "cr.flyte.org/flyteorg/flyteadmin:v1.1.77"
imagePullPolicy: "IfNotPresent"
command: ["/bin/sh", "-c"]
args:
Expand All @@ -935,7 +935,7 @@ spec:
- --config
- /etc/flyte/config/*.yaml
- serve
image: "cr.flyte.org/flyteorg/flyteadmin:v1.1.72"
image: "cr.flyte.org/flyteorg/flyteadmin:v1.1.77"
imagePullPolicy: "IfNotPresent"
name: flyteadmin
ports:
Expand Down Expand Up @@ -1036,7 +1036,7 @@ spec:
- /etc/flyte/config/*.yaml
- clusterresource
- run
image: "cr.flyte.org/flyteorg/flyteadmin:v1.1.72"
image: "cr.flyte.org/flyteorg/flyteadmin:v1.1.77"
imagePullPolicy: "IfNotPresent"
name: sync-cluster-resources
volumeMounts:
Expand Down Expand Up @@ -1089,7 +1089,7 @@ spec:
runAsUser: 1000
fsGroupChangePolicy: "OnRootMismatch"
containers:
- image: "cr.flyte.org/flyteorg/flyteconsole:v1.4.1"
- image: "cr.flyte.org/flyteorg/flyteconsole:v1.4.3"
imagePullPolicy: "IfNotPresent"
name: flyteconsole
envFrom:
Expand Down Expand Up @@ -1255,7 +1255,7 @@ spec:
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: "cr.flyte.org/flyteorg/flytepropeller:v1.1.64"
image: "cr.flyte.org/flyteorg/flytepropeller:v1.1.69"
imagePullPolicy: "IfNotPresent"
name: flytepropeller
ports:
Expand Down Expand Up @@ -1309,7 +1309,7 @@ spec:
labels:
app: flyte-pod-webhook
app.kubernetes.io/name: flyte-pod-webhook
app.kubernetes.io/version: v1.1.64
app.kubernetes.io/version: v1.1.69
annotations:
configChecksum: "ca9c48eb21423b6a8433aaa4a1451213cbbf3c1c5d856225b64f28e9a44cdb6"
spec:
Expand All @@ -1320,7 +1320,7 @@ spec:
serviceAccountName: flyte-pod-webhook
initContainers:
- name: generate-secrets
image: "cr.flyte.org/flyteorg/flytepropeller:v1.1.64"
image: "cr.flyte.org/flyteorg/flytepropeller:v1.1.69"
imagePullPolicy: "IfNotPresent"
command:
- flytepropeller
Expand All @@ -1343,7 +1343,7 @@ spec:
mountPath: /etc/flyte/config
containers:
- name: webhook
image: "cr.flyte.org/flyteorg/flytepropeller:v1.1.64"
image: "cr.flyte.org/flyteorg/flytepropeller:v1.1.69"
imagePullPolicy: "IfNotPresent"
command:
- flytepropeller
Expand Down
20 changes: 10 additions & 10 deletions deployment/eks/flyte_generated.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8724,7 +8724,7 @@ spec:
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: cr.flyte.org/flyteorg/flytepropeller:v1.1.64
image: cr.flyte.org/flyteorg/flytepropeller:v1.1.69
imagePullPolicy: IfNotPresent
name: webhook
volumeMounts:
Expand All @@ -8751,7 +8751,7 @@ spec:
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: cr.flyte.org/flyteorg/flytepropeller:v1.1.64
image: cr.flyte.org/flyteorg/flytepropeller:v1.1.69
imagePullPolicy: IfNotPresent
name: generate-secrets
volumeMounts:
Expand Down Expand Up @@ -8799,7 +8799,7 @@ spec:
- --config
- /etc/flyte/config/*.yaml
- serve
image: cr.flyte.org/flyteorg/flyteadmin:v1.1.72
image: cr.flyte.org/flyteorg/flyteadmin:v1.1.77
imagePullPolicy: IfNotPresent
name: flyteadmin
ports:
Expand Down Expand Up @@ -8846,7 +8846,7 @@ spec:
- /etc/flyte/config/*.yaml
- migrate
- run
image: cr.flyte.org/flyteorg/flyteadmin:v1.1.72
image: cr.flyte.org/flyteorg/flyteadmin:v1.1.77
imagePullPolicy: IfNotPresent
name: run-migrations
volumeMounts:
Expand All @@ -8863,7 +8863,7 @@ spec:
- flytesnacks
- flytetester
- flyteexamples
image: cr.flyte.org/flyteorg/flyteadmin:v1.1.72
image: cr.flyte.org/flyteorg/flyteadmin:v1.1.77
imagePullPolicy: IfNotPresent
name: seed-projects
volumeMounts:
Expand All @@ -8877,7 +8877,7 @@ spec:
- /etc/flyte/config/*.yaml
- clusterresource
- sync
image: cr.flyte.org/flyteorg/flyteadmin:v1.1.72
image: cr.flyte.org/flyteorg/flyteadmin:v1.1.77
imagePullPolicy: IfNotPresent
name: sync-cluster-resources
volumeMounts:
Expand All @@ -8897,7 +8897,7 @@ spec:
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: cr.flyte.org/flyteorg/flyteadmin:v1.1.72
image: cr.flyte.org/flyteorg/flyteadmin:v1.1.77
imagePullPolicy: IfNotPresent
name: generate-secrets
volumeMounts:
Expand Down Expand Up @@ -8951,7 +8951,7 @@ spec:
- envFrom:
- configMapRef:
name: flyte-console-config
image: cr.flyte.org/flyteorg/flyteconsole:v1.4.1
image: cr.flyte.org/flyteorg/flyteconsole:v1.4.3
name: flyteconsole
ports:
- containerPort: 8080
Expand Down Expand Up @@ -9002,7 +9002,7 @@ spec:
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: cr.flyte.org/flyteorg/flytepropeller:v1.1.64
image: cr.flyte.org/flyteorg/flytepropeller:v1.1.69
imagePullPolicy: IfNotPresent
name: flytepropeller
ports:
Expand Down Expand Up @@ -9270,7 +9270,7 @@ spec:
- /etc/flyte/config/*.yaml
- clusterresource
- sync
image: cr.flyte.org/flyteorg/flyteadmin:v1.1.72
image: cr.flyte.org/flyteorg/flyteadmin:v1.1.77
imagePullPolicy: IfNotPresent
name: sync-cluster-resources
volumeMounts:
Expand Down
Loading

0 comments on commit eba986b

Please sign in to comment.