Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added deployed prompts functionality with any amount of prompt variables #298

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open

added deployed prompts functionality with any amount of prompt variables #298

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
05f9b5c
added new prompt deployment endpoint
Oct 10, 2024
9d3348b
added prompt variable input to new ep
Oct 10, 2024
d3fb63e
Merge branch 'watson-developer-cloud:master' into deployed-prompt
nabil-ak Oct 15, 2024
605cd35
added the ability to set any prompt variables
nabil-ak Oct 15, 2024
3649966
added new action for deployed prompt ep
nabil-ak Oct 15, 2024
9d7458c
updated readme
nabil-ak Oct 15, 2024
0e0207a
updated readme
nabil-ak Oct 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 18 additions & 1 deletion integrations/extensions/starter-kits/language-model-watsonx/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,17 @@ The watsonx specification in the starter kit describes one endpoint and a few of
| Endpoint | Description |
| ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Generation | Used with watsonx text completion models such as `google/flan-ul2` and `google/flan-t5-xxl`. You provide text as a prompt, and it returns the text that follows that prompt. |
| Generation from a deployed prompt | Used with a deployed watsonx prompt. You provide the prompt deployment ID, and the prompt variables.

## Prerequisites

### Create an API key and a project ID
### Create an API key, project ID, and prompt deployment

1. Log in to [watsonx](https://dataplatform.cloud.ibm.com/wx/home?context=wx&apps=cos&nocache=true&onboarding=true&quick_start_target=watsonx) and [generate an API key](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/ml-authentication.html?context=cpdaas). Save this API key somewhere safe and accessible. You need this API key to set up the watsonx custom extension later.
1. To find your watsonx project id, go to [watsonx.ai](https://dataplatform.test.cloud.ibm.com/wx) and find Projects/<project-name> (this could be your `sandbox`, which is created for you by default). Click on the <project-name> link, then follow the Project's Manage tab (Project -> Manage -> General -> Details) to find the project id.

1. [Create your prompt](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-prompt-lab.html?context=wx&audience=wdp#creating-and-running-a-prompt) in Prompt Lab. Create the [prompt variables](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-prompt-variables.html?context=wx&audience=wdp#creating-prompt-variables) you need. Once your prompt is saved, [deploy your prompt](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/prompt-template-deploy.html?context=wx) and note your deployment ID.

### Create an assistant

This starter kit requires that you use the [new IBM watsonx Assistant](https://cloud.ibm.com/docs/watson-assistant?topic=watson-assistant-welcome-new-assistant).
Expand Down Expand Up @@ -63,6 +66,7 @@ The starter kit includes [a JSON file with sample actions](./watsonx-actions.jso
| ----------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Invoke watsonx Generation API | Connects to watson.ai with the selected model and the model input |
| Invoke watsonx Generation Stream API | Connects to watsonx.ai with the selected model and model input and streams the response back to the user. |
| Invoke watsonx Deployed Prompt API | Connects to the deployed prompt using the specified deployment ID and prompt variables. |
| Test model | Simple test action that asks what model, length, temperature, prompt you want, asks if you want to stream the response and then calls "Invoke watsonx Generation API" or "Invoke watsonx Generation Stream API" based on if streaming was required so the model can generate a response to the specified prompt. |
| No Action Matches | This is created by watsonx Assistant, but for this starter kit it is configured to trigger the "Invoke watsonx Generation Stream API" as a sub-action using the defaults and the user input. |

Expand Down Expand Up @@ -94,6 +98,19 @@ Here is an example of how to use the `Test model` action:

<img src="./assets/sample.png" width="300"/>

<br>

Here is an example of how to use the `Invoke watsonx Deployed Prompt API` action:

1. Set the `deployment-id`.
2. For `parameters.prompt_variables`:
- Use the expression: `{ "input": "1. What is your Prompt ", "name": "<your_next_prompt_variable_value>" }`
- Replace the key-value pairs with the appropriate variables from your deployed prompt.

This setup allows you to use a deployed watsonx prompt with any number of prompt variables, passing the user's input as a variable to generate a response for example.

<img src="./assets/deployed_prompt_example.png" width="800"/>

### Limitations

Adding large values for max_new_tokens & min_new_tokens can result in timeouts, if you see any timeouts please adjust the values of those tokens.
Binary file added ...tensions/starter-kits/language-model-watsonx/assets/deployed_prompt_example.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
277 changes: 263 additions & 14 deletions integrations/extensions/starter-kits/language-model-watsonx/watsonx-actions.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
"type": "action",
"valid": true,
"status": "Available",
"created": "2024-06-21T14:17:07.728Z",
"updated": "2024-07-11T17:16:52.580Z",
"created": "2024-10-10T08:10:42.574Z",
"updated": "2024-10-15T11:44:57.580Z",
"language": "en",
"skill_id": "fb7c641b-64b5-42cf-b57f-95f4b2d8c1d4",
"workspace": {
Expand Down Expand Up @@ -788,9 +788,9 @@
"type": "integration_interaction",
"method": "POST",
"internal": {
"spec_hash_id": "a0c4bc7edb01c8ff165651f82ee8f51064aae76f6a5f399126ad8e14dfe2469a",
"match_scenario": 5,
"catalog_item_id": "0aaa8850-fc91-41e5-81a1-ccd139799bb3"
"spec_hash_id": "3408d5415323ca392bb94b1fdfd03f6259f0ff1e35a65e5750e2740ba2908435",
"match_scenario": 10,
"catalog_item_id": "1ed41f7c-8373-4a95-af6b-21498918cdf0"
},
"request_mapping": {
"body": [
Expand Down Expand Up @@ -829,6 +829,18 @@
"skill_variable": "model_parameters_min_new_tokens"
},
"parameter": "parameters.min_new_tokens"
},
{
"value": {
"skill_variable": "model_parameters_stop_sequences"
},
"parameter": "parameters.stop_sequences"
},
{
"value": {
"skill_variable": "model_parameters_repetition_penalty"
},
"parameter": "parameters.repetition_penalty"
}
],
"query": [
Expand Down Expand Up @@ -991,9 +1003,9 @@
"type": "integration_interaction",
"method": "POST",
"internal": {
"spec_hash_id": "a0c4bc7edb01c8ff165651f82ee8f51064aae76f6a5f399126ad8e14dfe2469a",
"match_scenario": 5,
"catalog_item_id": "0aaa8850-fc91-41e5-81a1-ccd139799bb3"
"spec_hash_id": "5c85784df236387770379b6901bb0de4c0bd69368a0950a675f8e507c38196de",
"match_scenario": 10,
"catalog_item_id": "1ed41f7c-8373-4a95-af6b-21498918cdf0"
},
"request_mapping": {
"body": [
Expand All @@ -1015,6 +1027,12 @@
},
"parameter": "project_id"
},
{
"value": {
"skill_variable": "model_parameters_temperature"
},
"parameter": "parameters.temperature"
},
{
"value": {
"skill_variable": "model_parameters_max_new_tokens"
Expand All @@ -1029,9 +1047,15 @@
},
{
"value": {
"skill_variable": "model_parameters_temperature"
"skill_variable": "model_parameters_stop_sequences"
},
"parameter": "parameters.temperature"
"parameter": "parameters.stop_sequences"
},
{
"value": {
"skill_variable": "model_parameters_repetition_penalty"
},
"parameter": "parameters.repetition_penalty"
}
],
"query": [
Expand Down Expand Up @@ -1143,6 +1167,223 @@
"data_type": "any"
}
],
"next_action": "action_3200-4",
"topic_switch": {
"allowed_from": true,
"allowed_into": true,
"never_return": false
},
"disambiguation_opt_out": false
},
{
"type": "standard",
"steps": [
{
"step": "step_931",
"output": {
"generic": [
{
"values": [
{
"text_expression": {
"concat": [
{
"scalar": "What is your Prompt?"
}
]
}
}
],
"response_type": "text",
"selection_policy": "sequential"
}
]
},
"handlers": [],
"question": {
"free_text": true
},
"resolver": {
"type": "continue"
},
"variable": "step_931",
"next_step": "step_817"
},
{
"step": "step_817",
"output": {
"generic": []
},
"context": {
"variables": []
},
"handlers": [],
"resolver": {
"type": "callout",
"callout": {
"path": "/ml/v1/deployments/{deployment-id}/text/generation",
"type": "integration_interaction",
"method": "POST",
"internal": {
"spec_hash_id": "08f952f1e1a902a59aea69ad2b9df9a2e2d31b83fd62d1aefa68caab41fb4f5e",
"match_scenario": 10,
"catalog_item_id": "1ed41f7c-8373-4a95-af6b-21498918cdf0"
},
"request_mapping": {
"body": [
{
"value": {
"expression": "{\"input\":\"${step_931}\"}"
},
"parameter": "parameters.prompt_variables"
}
],
"path": [
{
"value": {
"scalar": "testing_prompt_deployment"
},
"parameter": "deployment-id"
}
],
"query": [
{
"value": {
"skill_variable": "watsonx_api_version"
},
"parameter": "version"
}
]
},
"result_variable": "step_817_result_2"
}
},
"variable": "step_817",
"next_step": "step_606"
},
{
"step": "step_606",
"output": {
"generic": [
{
"values": [
{
"text_expression": {
"concat": [
{
"scalar": ""
},
{
"skill_variable": "model_response"
}
]
}
}
],
"response_type": "text",
"selection_policy": "sequential"
}
]
},
"context": {
"variables": [
{
"value": {
"expression": "${step_817_result_2.body.results}[0].generated_text"
},
"skill_variable": "model_response"
}
]
},
"handlers": [],
"resolver": {
"type": "end_action"
},
"variable": "step_606",
"condition": {
"and": [
{
"eq": [
{
"variable": "step_817_result_2",
"variable_path": "success"
},
{
"scalar": true
}
]
},
{
"expression": "${step_817_result_2.body.results}.size() > 0"
}
]
},
"next_step": "step_212"
},
{
"step": "step_212",
"output": {
"generic": []
},
"context": {
"variables": [
{
"value": {
"expression": "null"
},
"skill_variable": "model_response"
}
]
},
"handlers": [],
"resolver": {
"type": "end_action"
},
"variable": "step_212"
}
],
"title": "Invoke watsonx Deployed Prompt API",
"action": "action_3200-4",
"boosts": [],
"handlers": [],
"condition": {
"intent": "action_3200_intent_45093-4"
},
"variables": [
{
"title": "",
"variable": "step_212",
"data_type": "any"
},
{
"title": "{variable}",
"variable": "step_606",
"data_type": "any"
},
{
"title": "No response",
"privacy": {
"enabled": false
},
"variable": "step_817",
"data_type": "any"
},
{
"privacy": {
"enabled": false
},
"variable": "step_817_result_2",
"data_type": "any"
},
{
"title": "What is your Prompt?",
"privacy": {
"enabled": false
},
"variable": "step_931",
"data_type": "any"
}
],
"next_action": "fallback",
"topic_switch": {
"allowed_from": true,
Expand Down Expand Up @@ -1707,6 +1948,14 @@
"intent": "action_3200_intent_45093-3",
"examples": []
},
{
"intent": "action_3200_intent_45093-4",
"examples": [
{
"text": "Invoke watsonx Deployed Prompt API"
}
]
},
{
"intent": "fallback_connect_to_agent",
"examples": [
Expand Down Expand Up @@ -1783,7 +2032,7 @@
"metadata": {
"api_version": {
"major_version": "v2",
"minor_version": "2021-11-27"
"minor_version": "2018-11-08"
}
},
"variables": [
Expand Down Expand Up @@ -1983,9 +2232,9 @@
"learning_opt_out": true
},
"description": "created for assistant 05c10d7d-336f-4d33-8cb3-5c53520d61ce",
"assistant_id": "577b3f36-98da-454c-becb-88970e3bec89",
"workspace_id": "fb7c641b-64b5-42cf-b57f-95f4b2d8c1d4",
"assistant_id": "0401c2c3-3bde-4465-8ac1-d77e093c000c",
"workspace_id": "c9ed31a7-e240-433d-bc69-74c42c2b6fd2",
"dialog_settings": {},
"next_snapshot_version": "1",
"environment_id": "782f075c-8d40-4396-95ca-70fb6b11a544"
"environment_id": "7a0b7575-6b2c-42c7-83de-c7fd1cfa0be5"
}
Loading