community[patch]: truncate zhipuai temperature and top_p parameters to [0.01, 0.99]
#20261
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Ignored Deployment
|
dosubot
bot
added
size:S
|
This PR follows up #16695 |
Congyuwang
changed the title
temperature and top_p parameters to [0.01, 0.99]temperature and top_p parameters to [0.01, 0.99]
|
Got the lint fixed. |
baskaryan
approved these changes
Apr 18, 2024
dosubot
bot
added
the
lgtm
|
Looks like the workflows needs approval too. Thx. |
auto-merge was automatically disabled
April 19, 2024 01:22
Head branch was pushed to by a user without write access
Congyuwang
commented
Apr 19, 2024
| """Truncate temperature and top_p parameters between [0.01, 0.99]. | ||
|
|
||
| ZhipuAI only support temperature / top_p between (0, 1) open interval, | ||
| so we truncate them to [0.01, 0.99]. |
There was a problem hiding this comment.
fixed typo in comment truncate
Congyuwang
commented
Apr 19, 2024
Comment on lines
-216
to
+230
| or you can use any finetune model of glm series. | ||
| Alternatively, you can use any fine-tuned model from the GLM series. |
There was a problem hiding this comment.
Fix some grammar in comment too.
hinthornw
pushed a commit
that referenced
this pull request
Apr 26, 2024
…rs to [0.01, 0.99] (#20261) ZhipuAI API only accepts `temperature` parameter between `(0, 1)` open interval, and if `0` is passed, it responds with status code `400`. However, 0 and 1 is often accepted by other APIs, for example, OpenAI allows `[0, 2]` for temperature closed range. This PR truncates temperature parameter passed to `[0.01, 0.99]` to improve the compatibility between langchain's ecosystem's and ZhipuAI (e.g., ragas `evaluate` often generates temperature 0, which results in a lot of 400 invalid responses). The PR also truncates `top_p` parameter since it has the same restriction. Reference: [glm-4 doc](https://open.bigmodel.cn/dev/api#glm-4) (which unfortunately is in Chinese though). --------- Co-authored-by: Bagatur <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
ZhipuAI API only accepts
temperatureparameter between(0, 1)open interval, and if0is passed, it responds with status code400.However, 0 and 1 is often accepted by other APIs, for example, OpenAI allows
[0, 2]for temperature closed range.This PR truncates temperature parameter passed to
[0.01, 0.99]to improve the compatibility between langchain's ecosystem's and ZhipuAI (e.g., ragasevaluateoften generates temperature 0, which results in a lot of 400 invalid responses). The PR also truncatestop_pparameter since it has the same restriction.Reference: glm-4 doc (which unfortunately is in Chinese though).