Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improvement] kbcli support requesting gpu resources #5608

Open
lynnleelhl opened this issue Oct 25, 2023 · 1 comment
Open

[Improvement] kbcli support requesting gpu resources #5608

lynnleelhl opened this issue Oct 25, 2023 · 1 comment
Assignees
Labels
good first issue Denotes an issue ready for a new contributor, according to the "help wanted" guidelines. kind/enhancement New feature or request Stale

Comments

@lynnleelhl
Copy link
Contributor

llm with vllm framework need gpu to run, kbcli should support setting gpu resource request in command line, here's the gpu request example in yaml

apiVersion: apps.kubeblocks.io/v1alpha1
kind: Cluster
metadata:
  name: bc
spec:
  clusterDefinitionRef: vllm # ref clusterdefinition.name
  clusterVersionRef: vllm-baichuan2-13b
  terminationPolicy: Delete
  componentSpecs:
  - name: vllm
    componentDefRef: vllm
    monitor: false
    replicas: 2
    resources:
      requests:
        nvidia.com/gpu: 1
      limits:
        nvidia.com/gpu: 1
@lynnleelhl lynnleelhl added the kind/enhancement New feature or request label Oct 25, 2023
@lynnleelhl lynnleelhl assigned nayutah and ldming and unassigned nayutah Oct 25, 2023
Copy link

This issue has been marked as stale because it has been open for 30 days with no activity

@github-actions github-actions bot added the Stale label Nov 27, 2023
@free6om free6om added the good first issue Denotes an issue ready for a new contributor, according to the "help wanted" guidelines. label Dec 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Denotes an issue ready for a new contributor, according to the "help wanted" guidelines. kind/enhancement New feature or request Stale
Projects
None yet
Development

No branches or pull requests

4 participants