Setting the API key for Cohere and Anthropic per instance

[anirbanbasu]

I have a use-case where I would like to change the API key for either a Cohere LLM or an Anthropic LLM on a per instance basis. However, I do not want to create new instances of the LLM objects.

With LLM objects such as Groq, Open AI and others, it is possible to call .api_key = MY_KEY on the respective LLM object. However, with Anthropic and Cohere (and perhaps, some others that I have not tried to use), I get an error such as (for Cohere):

AttributeError: 'Cohere' object has no attribute 'api_key'

Why is that even though I can set api_key in the constructor of Cohere? Is this a bug that should be fixed? Or, is this intended/expected behaviour?

For reference, I am using the following imports.

from llama_index.llms.groq import Groq
from llama_index.llms.anthropic import Anthropic
from llama_index.llms.cohere import Cohere
from llama_index.llms.openai import OpenAI

[dosubot[bot]]

The inability to set the api_key attribute on a Cohere LLM object after instantiation is intended behavior, not a bug. This is because the api_key is used during the initialization of the _client and _aclient attributes, which are instances of cohere.Client and cohere.AsyncClient, respectively. These clients are created in the __init__ method and require the api_key at that time. Changing the api_key after instantiation would not update these client instances ^[1].

For the Anthropic class, I wasn't able to find specific information within the repository, but it is likely that it follows a similar pattern where the api_key is used during initialization and cannot be changed afterward.

If you need to change the api_key for these LLMs, you would need to create a new instance of the LLM object with the new api_key.

To continue talking to Dosu, mention @dosu.

[anirbanbasu]

@dosu Thanks for your explanation. In general, then, is it okay to set the api_key on the LLM object after instantiation for cases where it works, e.g., Groq and Open AI? Or, things may change later and it is better to create a new instance of the LLM object anyway?

[dosubot[bot]]

It is okay to set the api_key on the LLM object after instantiation for cases where it works, such as Groq and OpenAI. The provided documentation shows that you can set the API key at a per-instance level, indicating that creating a new instance is not strictly necessary ^[1].

To continue talking to Dosu, mention @dosu.

[teaghan]

For Anthropic models, I believe this works:

from llama_index.llms.anthropic import Anthropic

my_api_key = 'sk-***'

llm_model = Anthropic()
llm_model._client.api_key = my_api_key
llm_model._aclient.api_key = my_api_key