Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvement Request: Add parameter to support decoding_method="sample" #248

Open
hirokik1 opened this issue Feb 16, 2024 · 0 comments
Open

Comments

@hirokik1
Copy link

Hello,

I have been using the watsonx-openapi.json file for creating an extension for the watsonx Assistant, and I would like to express my gratitude for making it available.
integrations/extensions/starter-kits/language-model-watsonx/watsonx-openapi.json

I am writing to submit a request for an improvement.
I would appreciate it if you could add the following parameters to the parameters section, so that it becomes possible to execute the API with the decoding_method set to "sample".
The description below is quoted from the BAM explanation site. I have tested this in my environment and confirmed it works.

"decoding_method": {
  "type": "string",
  "description": "Represents the strategy used for picking the tokens during generation of the output text. Options are greedy and sample. Value defaults to sample if not specified.",
  "example": "greedy"
},
"top_k": {
  "type": "number",
  "description": "The number of highest probability vocabulary tokens to keep for top-k-filtering. Only applies for sampling mode, with range from 1 to 100. When decoding_strategy is set to sample, only the top_k most likely tokens are considered as candidates for the next generated token.",
  "example": "50"
},
"top_p": {
  "type": "number",
  "description": "Similar to top_k except the candidates to generate the next token are the most likely tokens with probabilities that add up to at least top_p. The valid range is 0.0 to 1.0 where 1.0 is equivalent to disabled and is the default. Also known as nucleus sampling.",
  "example": "1"
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

1 participant