Support more LLMs

[Thore1954]

Is there a plan to support LLMs other than those from OpenAI? There is huggingface-vscode that works with HF models but it doesn't have a chat interface.
Does anyone know of a similar plugin that works with LLMs on HF?

[Christopher-Hayes]

Right now, it's models that mimic OpenAI's API. I believe there are some GitHub projects that "proxy" other models to use OpenAI's API, which would allow this extension to use other models. Long-term it would be great to support all kinds of models, I don't see that happening in the near future, because that kind of refactoring will take time. But, we might start using Vercel's new ai library soon, which has "First-class support for LangChain and OpenAI, Anthropic, Cohere and Hugging Face".

[Thore1954]

So basically I can wrap HF's Inference Endpoints in a server that mimics OpenAI's API and set it in the chatgpt.gpt3.apiBaseUrl config? Didn't know that. Thanks.

[Christopher-Hayes]

Yeah, that's the idea. I think some people are doing that now for Claude. That config is being used a lot by people who geographically are blocked from accessing api.openai.com and instead use a proxy server. I haven't personally done anything with other models, so I can't offer much more detail than that.