Should we update assistant's GPT versions?

[mikaelsouza]

GPT-3.5 Turbo and GPT-4 Turbo have newer, cheaper versions available now.

Jan 25, 2024 post: https://openai.com/blog/new-embedding-models-and-api-updates

It looks like we can update both gpt-3.5-turbo and gpt-4-turbo to these improved versions to get better results while expending less money.

As far as I explored, it seems we just need to change the file crates/assistant/src/assistant_settings.rs to point to these newer models.

My question, then, is if someone from the project can confirm this is a desired change, if it is, I'd like to change this as my first contribution to the project. :)

[iamnbutler]

Sure, I'd feel free just to put up a PR!

We haven't done much work on the assistant recently but things should pick up soon, hopefully we'll be able to explore more options here as well.

At some point we'll want this to be customizable, but for now just adding the updated available versions is probably fine.

[mikaelsouza]

It looks like tiktoken-rs has not been updated yet to support these new versions. The consequence is that, when we use one of these newer models, tiktoken reports the incorrect context size for the model. I'll open a PR there, and I'll be back adding these new changes as soon as these get merged.

[mikaelsouza]

The PR is open:

zurawiki/tiktoken-rs#59

We just need to wait to check if everything is correctly addressed, then, after a new release, we can update the models in Zed. :^)

[xpe]

@mikaelsouza How do you feel about closing this discussion? Seems like the problem is solved.