Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KEP-2170: Deploy JobSet in kubeflow-system namespace #2388

Merged
merged 6 commits into from
Jan 27, 2025
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Create third-party dir for JobSet
Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
  • Loading branch information
andreyvelich committed Jan 26, 2025
commit 6021349dc3740d979e7acfe54647f57d1f3b35d2
20 changes: 3 additions & 17 deletions manifests/v2/overlays/manager/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,30 +10,16 @@ resources:
- ../../base/manager
- ../../base/rbac
- ../../base/webhook
- https://github.com/kubernetes-sigs/jobset/releases/download/v0.7.2/manifests.yaml
- ../../third-party/jobset # Comment this line if JobSet is installed on the Kubernetes cluster.

# Update the Kubeflow Training manager image tag.
images:
- name: kubeflow/training-operator-v2
newTag: latest

# Disable postfix for Secret and ConfigMap
generatorOptions:
disableNameSuffixHash: true

# Secret for the Kubeflow Training webhook.
secretGenerator:
- name: training-operator-v2-webhook-cert
namespace: kubeflow-system

# Config for the JobSet manager.
# TODO (andreyvelich): Remove this when this is released: https://github.com/kubernetes-sigs/jobset/issues/720
configMapGenerator:
- files:
- jobset_manager_config.yaml
name: jobset-manager-config

# Add required patches.
patchesStrategicMerge:
- patches/jobset_remove_namespace.yaml # Remove namespace from the JobSet release manifests.
- patches/jobset_config_patch.yaml # Add custom manager config to the JobSet.
options:
disableNameSuffixHash: true
19 changes: 19 additions & 0 deletions manifests/v2/third-party/jobset/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- https://github.com/kubernetes-sigs/jobset/releases/download/v0.7.2/manifests.yaml

# Config for the JobSet manager.
# TODO (andreyvelich): Remove this when this is released: https://github.com/kubernetes-sigs/jobset/issues/720
configMapGenerator:
- name: jobset-manager-config
files:
- jobset_manager_config.yaml
options:
disableNameSuffixHash: true

# Add required patches.
patchesStrategicMerge:
- patches/jobset_remove_namespace.yaml # Remove namespace from the JobSet release manifests.
Comment on lines +16 to +17
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shall we also patch those namespace related configurations outside of the .metadata field? Like: https://github.com/kubeflow/training-operator/pull/2382/files#r1914094979

It seems that we also need to patch those webhook configurations to ensure jobsetworking correctly:)

Copy link
Member Author

@andreyvelich andreyvelich Jan 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed that even without this config, the kustomize correctly patches the JobSet's manifests with:

  clientConfig:
    service:
      name: jobset-webhook-service
      namespace: kubeflow-system
      path: /validate--v1-pod

I think, it works without this configuration since the JobSet manifests includes this service which tells kustomize that namespace in clientConfig needs to be patched.

apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: webhook
    app.kubernetes.io/created-by: jobset
    app.kubernetes.io/instance: webhook-service
    app.kubernetes.io/managed-by: kustomize
    app.kubernetes.io/name: service
    app.kubernetes.io/part-of: jobset
  name: jobset-webhook-service
  namespace: jobset-system
spec:
  ports:
    - port: 443
      protocol: TCP
      targetPort: 9443
  selector:
    control-plane: controller-manager

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM!

- patches/jobset_config_patch.yaml # Add custom manager config to the JobSet.
Loading