Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: add support for o1 models in openai and azure #368

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

feat: add support for o1 models in openai and azure #368

wants to merge 1 commit into from

Conversation

sheldonhull
Copy link

Add support for OpenAI o1 models by using max_completion_tokens instead of max_tokens.

  • mods.go

    • Add a check in the startCompletionCmd function to determine if the model is an o1 model and set the max_completion_tokens parameter accordingly.

  • config.go

    • Add a new field MaxCompletionTokens to the Config struct to store the value for the max_completion_tokens parameter.

  • config_template.yml

    • Add entries for o1-preview and o1-mini models under the openai section with max-input-chars set to 128000.
    • Add aliases for o1-preview and o1-mini models.
    • Add entries for o1-preview and o1-mini models under the azure section with max-input-chars set to 128000.
    • Add aliases for o1-preview and o1-mini models under the azure section.

@sheldonhull
feat: add support for o1 models in openai and azure
f9560d4 
Add support for OpenAI o1 models by using `max_completion_tokens` instead of `max_tokens`.

* **mods.go**
  - Add a check in the `startCompletionCmd` function to determine if the model is an o1 model and set the `max_completion_tokens` parameter accordingly.

* **config.go**
  - Add a new field `MaxCompletionTokens` to the `Config` struct to store the value for the `max_completion_tokens` parameter.

* **config_template.yml**
  - Add entries for `o1-preview` and `o1-mini` models under the `openai` section with `max-input-chars` set to 128000.
  - Add aliases for `o1-preview` and `o1-mini` models.
  - Add entries for `o1-preview` and `o1-mini` models under the `azure` section with `max-input-chars` set to 128000.
  - Add aliases for `o1-preview` and `o1-mini` models under the `azure` section.
@sheldonhull
Copy link
Author

Note, this is placeholder PR to jump start via github workspaces spec, but I haven't gone through and changed 32768 as max tokens and some other steps. Figured I'd get it up as a draft to get it on the radar and can improve soon. So take this as a true draft for investigating fix for #367

@sheldonhull
Copy link
Author

@caarlos0 so overall this seemed correct as long as I fix tokens? I generated this and didn't have time to come back round then noticed your feedback was applied. I can try and get this wrapped up today the. If you have no issues with approach.

@caarlos0
Copy link
Member

caarlos0 commented Nov 7, 2024

@caarlos0 so overall this seemed correct as long as I fix tokens? I generated this and didn't have time to come back round then noticed your feedback was applied. I can try and get this wrapped up today the. If you have no issues with approach.

yes, seems correct to me, at least at first sight. Happy to help test/etc or if you need anything 🙏🏻

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@caarlos0 caarlos0 caarlos0 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@sheldonhull @caarlos0