AgentQnA helm chart deploy update #837
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sync latest changes with GenAIExamples. Added cpu deployment with smaller model. Updated README with detailed instructions. Support using PVC for passing tools configuration. Fix minor issues. Signed-off-by: Dolpher Du <[email protected]>
eero-t
reviewed
Feb 27, 2025
|
|
||
| Note that this is an example to demonstrate how agent works and tested with prepared data and questions. Using different datasets, models and questions may get different results. | ||
|
|
||
| Agent usually requires larger models to performance better, we used Llama-3.3-70B-Instruct for test, which requires 4x Gaudi devices for local deployment. |
There was a problem hiding this comment.
Typo:
Suggested change
| Agent usually requires larger models to performance better, we used Llama-3.3-70B-Instruct for test, which requires 4x Gaudi devices for local deployment. | |
| Agent usually requires larger models to perform better, we used Llama-3.3-70B-Instruct for test, which requires 4x Gaudi devices for local deployment. |
I guess it could be run (slowly) also on CPU with enough memory?
|
|
||
| Agent usually requires larger models to performance better, we used Llama-3.3-70B-Instruct for test, which requires 4x Gaudi devices for local deployment. | ||
|
|
||
| With helm chart, we also provided option with smaller model(Meta-Llama-3-8B-Instruct) with compromised performance on Xeon CPU only environment for you to try. |
There was a problem hiding this comment.
Suggested change
| With helm chart, we also provided option with smaller model(Meta-Llama-3-8B-Instruct) with compromised performance on Xeon CPU only environment for you to try. | |
| With helm chart, we also provided option with smaller model (Meta-Llama-3-8B-Instruct) with compromised performance on Xeon CPU only environment for you to try. |
|
|
||
| ## Deploy | ||
|
|
||
| helm install agentqna oci://ghcr.io/opea-project/charts/agentqna --set global.HUGGINGFACEHUB_API_TOKEN=${HUGGINGFACEHUB_API_TOKEN} --set tgi.enabled=True | ||
| The Deployment includes preparing tools and sql data. |
There was a problem hiding this comment.
Suggested change
| The Deployment includes preparing tools and sql data. | |
| The Deployment includes preparing tools and SQL data. |
|
|
||
| A volume is required to put tools configuration used by agent, and the database data used by sqlagent. | ||
|
|
||
| We'll use hostPath in this readme, which is convenient for single worker node deployment. PVC is recommended in a bigger cluster. If you want to use a PVC, comment out the `toolHostPath` and replace with `toolPVC` in the values.yaml. |
There was a problem hiding this comment.
Suggested change
| We'll use hostPath in this readme, which is convenient for single worker node deployment. PVC is recommended in a bigger cluster. If you want to use a PVC, comment out the `toolHostPath` and replace with `toolPVC` in the values.yaml. | |
| We'll use hostPath in this readme, which is convenient for single worker node deployment. PVC is recommended in a bigger cluster. If you want to use a PVC, comment out the `toolHostPath` and replace with `toolPVC` in the `values.yaml`. |
|
|
||
| We'll use hostPath in this readme, which is convenient for single worker node deployment. PVC is recommended in a bigger cluster. If you want to use a PVC, comment out the `toolHostPath` and replace with `toolPVC` in the values.yaml. | ||
|
|
||
| Create the directory /mnt/tools in the worker node, which is the default in values.yaml. We use the same directory for all 3 agents for easy configuration. |
There was a problem hiding this comment.
Suggested change
| Create the directory /mnt/tools in the worker node, which is the default in values.yaml. We use the same directory for all 3 agents for easy configuration. | |
| Create the directory `/mnt/tools` in the worker node, which is the default in `values.yaml`. We use the same directory for all 3 agents for easy configuration. |
| OPENAI_API_KEY: EMPTY | ||
| model: "YourModel" | ||
| # Use OpenAI KEY | ||
| # llm_engine: openai |
There was a problem hiding this comment.
Why these are commented out?
Comment on lines
+18
to
+22
| OPENAI_API_KEY: EMPTY | ||
| model: "YourModel" | ||
| # Use OpenAI KEY | ||
| # llm_engine: openai | ||
| # OPENAI_API_KEY: YourOpenAIKey |
There was a problem hiding this comment.
These extra key comments are redundant for all of these 3 subcharts.
| @@ -6,6 +6,24 @@ | |||
|
|
|||
There was a problem hiding this comment.
Just in case:
Suggested change
| vllm: | |
| enabled: false | |
| @@ -6,6 +6,15 @@ | |||
|
|
|||
There was a problem hiding this comment.
Just in case:
Suggested change
| tgi: | |
| enabled: false | |
Comment on lines
+30
to
+31
| # Uncomment this if you have an tool configuration file | ||
| tools: /home/user/comps/agent/src/tools/custom_tools.yaml |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Sync latest changes with GenAIExamples.
Added cpu deployment with smaller model.
Updated README with detailed instructions.
Support using PVC for passing tools configuration. Fix minor issues.
Description
Update agentqna helm charts
Issues
Closed #827
Closed #798
Closed #783
Example 1524
Example 1523
Type of change
List the type of change like below. Please delete options that are not relevant.
Dependencies
List the newly introduced 3rd party dependency if exists.
Tests
Manual tested deploy and helm test with gaudi-values.yaml and cpu-values.yaml.