Add support for Llama2, Palm, Cohere, Anthropic, Replicate Models - using litellm

[ishaan-jaff]

This PR adds support for models from all the above mentioned providers using https://github.com/BerriAI/litellm/

Here's a sample of how it's used:

from litellm import completion, acompletion

## set ENV variables
# ENV variables can be set in .env file, too. Example in .env.example
os.environ["OPENAI_API_KEY"] = "openai key"
os.environ["COHERE_API_KEY"] = "cohere key"

messages = [{ "content": "Hello, how are you?","role": "user"}]

# openai call
response = completion(model="gpt-3.5-turbo", messages=messages)

# llama2 call
model_name = "replicate/llama-2-70b-chat:2c1608e18606fad2812020dc541930f2d0495ce32eee50074220b87300bc16e1"
response = completion(model_name, messages)

# cohere call
response = completion("command-nightly", messages)

# anthropic call
response = completion(model="claude-instant-1", messages=messages)

[ishaan-jaff]

@yyiilluu can you take a look at this PR when possible😊?
Happy to add more docs/tests if this initial commit looks good

[ishaan-jaff]

This should also remove the need to separately maintain the HF integration in AutoChain

[yyiilluu]

a few questions, does litellm an API wrapper around all different LLM providers by standardizing the outputs to match what OpenAI has? who is paying of the hosting cost for HF model, is litellm paying replicate for hosting?

[ishaan-jaff]

does litellm an API wrapper around all different LLM providers by standardizing the outputs to match what OpenAI has

Yes, outputs match OpenAI, more info here: https://docs.litellm.ai/docs/completion/output

who is paying of the hosting cost for HF model, is litellm paying replicate for hosting?

You pay, you must set your API keys in your .env or pass it in completion or as litellm.api_key

[yyiilluu]

got you, i have to say i am a bit worried about this as there could be changes from OpenAI but not picked up by LiteLLM, which cause AutoChain not able to be benefited from it

[ishaan-jaff]

liteLLM is an actively maintained library and always uses the latest versions of openai
https://github.com/BerriAI/litellm/blob/main/pyproject.toml#L11

However if adding new LLMs is currently not a priority for AutoChain ,we can close this PR

[ishaan-jaff]

Hi @yyiilluu wanted to follow up

What was litellm missing to be useful to you? Any feedback here would be helpful