feat (packages/openai-compatible): Base for OpenAI-compatible providers. #3812
Conversation
shaper
force-pushed
the
shaper/pr/provider-openai-compatible
branch
from
d53c857
to
90b9c52
Compare
packages/openai-compatible/src/openai-compatible-completion-language-model.ts
Outdated
Show resolved
Hide resolved
shaper
force-pushed
the
shaper/pr/provider-openai-compatible
branch
from
90b9c52
to
e32f8db
Compare
shaper
force-pushed
the
shaper/pr/provider-openai-compatible
branch
from
e32f8db
to
2978847
Compare
| import { createOpenAICompatible } from '@ai-sdk/openai-compatible'; | ||
| ``` | ||
|
|
||
| ## Example |
There was a problem hiding this comment.
There should be a section on creating a custom provider leveraging this package. I'm planning to add that once we have the TogetherAI one done.
shaper
force-pushed
the
shaper/pr/provider-openai-compatible
branch
from
21e1cc0
to
c7ecf2f
Compare
There was a problem hiding this comment.
I wasn't able to find a model on together.ai that supported image inputs after 30m of trying a few that seemed like they might. So I switched to OpenAI and validated it's operational. I will try with other companies/models later as we generalize the error schema and so forth to make it easier to test others.
shaper
force-pushed
the
shaper/pr/provider-openai-compatible
branch
from
c7ecf2f
to
83679e8
Compare
|
|
||
| async function main() { | ||
| const togetherai = createOpenAICompatible({ | ||
| apiKeyEnvVarName: 'TOGETHER_AI_API_KEY', |
There was a problem hiding this comment.
let's completely remove the concept of env variables for openai compatible. it's an unnecessary complexity in this special case, users can just pass the env property and we have less API surface
There was a problem hiding this comment.
(it it only there for convenience to begin with, and here convenience becomes complexity)
There was a problem hiding this comment.
I'm wondering if we should go even further and remove the api key completely. some providers don't need api keys, and other have different header settings. instead it should be possible to send in default headers.
|
|
||
| const { text } = await generateText({ | ||
| model: createOpenAICompatible({ | ||
| apiKeyEnvVarName: 'EXAMPLE_API_KEY', |
There was a problem hiding this comment.
directly pass env property instead
| Description of the API key environment variable (for use in error messages). | ||
| */ | ||
| apiKeyEnvVarDescription?: string; | ||
|
|
There was a problem hiding this comment.
lets remove api key related code completely. instead users can provide the key via headers. reason being that some providers have no api keys and there are different ways to send it via headers
| const getHeaders = () => ({ | ||
| Authorization: `Bearer ${loadApiKey({ | ||
| apiKey: options.apiKey, | ||
| environmentVariableName: options.apiKeyEnvVarName ?? '', | ||
| description: options.apiKeyEnvVarDescription ?? '', | ||
| })}`, | ||
| ...options.headers, | ||
| }); |
There was a problem hiding this comment.
we can get rid of this and just use options.headers
Breakout from #3781. Follow-up items:
json_schemaandstructuredOutputsinopenai-compatible-chat-language-model.tscontent)