feat: show available models list in settings

[adietrichs]

Motivation

Currently, the list of supported models is hard-coded. That has several downsides:

• not all users / API keys have access to all hard-coded models (see fix: error handling if user has no access to gpt4 yet #26)
• not all available chat models supported, like the new gpt-3.5-turbo-16k
• no access to specific (older) model versions like gpt-4-0314

Solution

• add a new utils/models.ts with a getAvailableChatModels(apiKey) function
• fetch available models every time the API key changes (including on app load)
• update model setting if current model no longer available
• show list of models on settings modal
• consider model list load for isAnythingLoading

Screenshot of an exemplary model list:

Checklist

• Tested in Chrome
• Tested in Safari

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated (UTC)
flux	✅ Ready (Inspect)	Visit Preview	Jun 15, 2023 1:18am

[adietrichs]

Note that I have little js/ts and basically no React experience, so while this looks like it works, it might not follow best practices.

[transmissions11]

not sure we should include availableModels === null here as this is more about things being saved

[adietrichs]

I really like to have the spinner continue to spin until that request finishes. Added a distinction between isAnythingSaving and isAnythingLoading to address that.

[transmissions11]

this looks fantastic, nice work. only notes:

• should we maybe filter out the fixed version models (the ones that end with -[number])? kinda clogs up the UI and i dont know anyone who needs them outside of like big enterprise

• still not picking up GPT-4-32k on my account, idk where they surface that its accessible

[transmissions11]

should we maybe filter out the fixed version models (the ones that end with -[number])? kinda clogs up the UI and i dont know anyone who needs them outside of like big enterprise

I guess this is one case why. The latest capabilities arent on the stable gpt-4 endpoint, only on the fixed version atm

[adietrichs]

should we maybe filter out the fixed version models (the ones that end with -[number])?

I guess this is one case why. The latest capabilities arent on the stable gpt-4 endpoint, only on the fixed version atm

I addressed that in the best way I could come up with, by having a hard-coded list of models to exclude. That way, usually only the stable models would be listed, but in moments like now, these additional new models show up automatically. I prefer the default to be for new versions to be shown, so app updates are only required to de-clutter, not to make new functionality accessible.

still not picking up GPT-4-32k on my account, idk where they surface that its accessible

This is odd. I am unfortunately still on the waitlist for that one, so can't test myself (of course feel free to dm me an API key "for testing" 🙂). But I did see inconsistent behavior around that yesterday myself, when it briefly listed gpt-4-32k-0613 as available for me. So maybe this is just a temporary problem on their end. Otherwise not sure what to do here.

[transmissions11]

I addressed that in the best way I could come up with, by having a hard-coded list of models to exclude

hmm i appreciate this but probably too much of a hassle to maintain, can just leave as is imo

This is odd. I am unfortunately still on the waitlist for that one, so can't test myself (of course feel free to dm me an API key "for testing" 🙂). But I did see inconsistent behavior around that yesterday myself, when it briefly listed gpt-4-32k-0613 as available for me. So maybe this is just a temporary problem on their end. Otherwise not sure what to do here.

dming you a 32k key via twitter so you can take a look — if its an issue on their end can always just edit the stored model json in localstorage in the meantime i guess?

[adietrichs]

hmm i appreciate this but probably too much of a hassle to maintain, can just leave as is imo

Okay, reverted back to the version without hidden models.

dming you a 32k key via twitter so you can take a look — if its an issue on their end can always just edit the stored model json in localstorage in the meantime i guess?

When testing with that key now, the gpt-4-32k model shows up correctly (see screenshot below). Can you check whether it still misbehaves for you? If so, maybe re-generating your key would help... But regarding your suggestion, right now editing the settings in localstorage would specifically not help, as the PR has logic to fall back to the default model in case the model from settings is not in the list.

[transmissions11]

Okay, reverted back to the version without hidden models.

🙏

When testing with that key now, the gpt-4-32k model shows up correctly (see screenshot below). Can you check whether it still misbehaves for you?

oh weird yeah works for me as well now... 🤷

But regarding your suggestion, right now editing the settings in localstorage would specifically not help, as the PR has logic to fall back to the default model in case the model from settings is not in the list.

mm right gotcha.

[transmissions11]

alright this lgtm! great work @adietrichs