Add Google Gemini API Support #1134
Replies: 64 comments 1 reply
-
|
import os llm = ChatGoogleGenerativeAI(model="gemini-pro",verbose = True,temperature = 0.1,google_api_key="GEMINI-API-KEY") Define your agents with roles and goalsresearcher = Agent( writer = Agent( task1 = Task( task2 = Task( Instantiate your crew with a sequential processcrew = Crew( Get your crew to work!result = crew.kickoff() print("######################") ##Thank you for the update |
Beta Was this translation helpful? Give feedback.
-
|
@janda-datascience Thank you so much! Really excited to get CrewAI working with Google Gemini. I ran into one issue with your code, it appers /crewai/agent.py is still looking for openAI and its key for its functions and won't run without an OpenAI key with the error: "Did not find openai_api_key, please add an environment variable I assume settings need to be changed in /crewai/agent.py to point to the Gemini Pro API instead of OpenAI for all agent functions, so it doesn't bill the OpenAI API? It looks like this is the section, could you please help provide the proper updates to agent.py . In contrast, If I just add the OpenAI API, it start running agent tasks and billing me on openai api. Really appreciate your help. agent.py section to modify? (Lines 57-60 in agent.py) llm: Optional[Any] = Field( Error: agent.py calling for OpenAI still: File "/home/aivirtual/miniconda3/envs/crewai/lib/python3.10/site-packages/langchain_core/load/serializable.py", line 107, in init |
Beta Was this translation helpful? Give feedback.
-
|
@mindwellsolutions by default the agent uses gpt-4, if you want to use your gemini model you can provide the gemini llm inside the agent definition: eg: |
Beta Was this translation helpful? Give feedback.
-
|
@punitchauhan771 Thank you so much, that worked! It successfully runs CrewAI completely free with Gemini Pro API! Going to spend some time testing it out. Really appreciate the help here. I had to add one small update to the solution punitchauhan provided to get it working, thought I'd share it with everyone. You need to define the google_api_key directly within the llm = line of code. (Working Update to llm = code for gemini api key) Last Question: Google Gemini API (Free) has a limit of 60 API uses per minute. Is there a way to throttle the agent activity, so it doesn't perform more than 59 API requests per minute, so we can avoid going over free limits when dispatching large agent swarms? Really appreciate everything, ty. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @mindwellsolutions, at the end of the agent activity it will give you some response like : hope this helps. |
Beta Was this translation helpful? Give feedback.
-
|
@punitchauhan771 Thank you and everyone for the help, such an amazing community. This would help a lot, but I'm using Google Gemini API rather than OpenAI with from langchain_google_genai import ChatGoogleGenerativeAI. Is there a callback for ChatGoogleGenerativeAI similar to the OpenAI callback you posited "from langchain.callbacks import get_openai_callback". Really appreciate the help. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @mindwellsolutions the solution I provided works for Gemini as well,though it doesn't count tokens 🙂. |
Beta Was this translation helpful? Give feedback.
-
|
@punitchauhan771 Thanks again. Unfortunately using your solution after 2 agents complete all their tasks successfully with Gemini API its still showing "Successful Request: 0". As you mentioned simply being able to monitor the amount of Successful Requests is all Gemini Pro (Free Tier) needs since there are no costs to track associated with token usage :) The Output I get after Gemini clearly performs multiple tasks: |
Beta Was this translation helpful? Give feedback.
-
|
@mindwellsolutions if possible can you provide me the code snippet? |
Beta Was this translation helpful? Give feedback.
-
|
@punitchauhan771 Actually. I closed the terminal window in VS Code and ran it again and it worked perfect. Really appreciate all your help, you've gotten everything running perfectly for me. Ty |
Beta Was this translation helpful? Give feedback.
-
|
The performance of the Gemini Pro model using tools is not good. There are big problems with the logic and results of using the tools, and the final results are often hallucinatory. |
Beta Was this translation helpful? Give feedback.
-
|
@edisonzf2020 Thanks for your comment. I got time to test out gemini api in crewai further over the weekend and as you mentioned it seems to be having issues using tools like duckduckgosearch. I tested openai API and Ollama (zephyr) and those both worked perfectly, while it looks like Gemini is pulling responses from its internal knowledgebase rather than relaying the data from DuckDuckGo. Are there any possible fixes moving forward to get Gemini to play well with CrewAI? The scalability of using a free API for personal use will provide incredible power of scalability without having to run local models on local GPU resources. |
Beta Was this translation helpful? Give feedback.
-
|
Hey folks, catching up to this issue! great comments, so glad you were able to get Gemini working, we are adding new docs that will have instructions for all the major models so stay tuned for that. I'll do some testing with Gemini models specifically to see how we could make that better! |
Beta Was this translation helpful? Give feedback.
-
|
@joaomdmoura Thank you. This is incredibly appreciated. We realized that Gemini API has significant problems currently using CrewAI's tools like DuckDuckGoSearch. It appears none of the data from the tools make it back to Gemini, and Gemini always generates answers from its internal knowledge base rather than the research done by the tool. If there is a way to fix this so Gemini works like other LLMs that would be amazing. I've been using Zephyr 7b as a local model and that runs well, but the power of decent size scalability of swarms that would require significant power to run locally, can be run from any device using Gemini's free API - which is going to be extremely valuable. Especially if each Agent is assigned it's own Free Gemini API key in concert with each other, the ability to built large API swarms for free will be significant. Thanks for all that you do. Really loving the functionality of CrewAI. |
Beta Was this translation helpful? Give feedback.
-
Hi @mindwellsolutions, response: when i tried google search the response was: and when I tried duckduckgoSearch the response was: also when i tried the same using langchain agent, the response were same, but when i just used the basic llm, the response was: Also it doesn't allow questionable prompt as mentioned in the gemini doc: |
Beta Was this translation helpful? Give feedback.
-
|
I replaced @tool('DuckDuckGoSearch')
def search_tool(search_query: str):
"""Search the web for information on a given topic"""
return DuckDuckGoSearchRun().run(search_query)to be: @tool('DuckDuckGoSearch')
def search_tool(search_query: str):
"""Search the web for information on a given topic"""
return DuckDuckGoSearchResults().run(search_query)Here is a screenshot of the output: the log as text: As you can see it retrieved the URL. |
Beta Was this translation helpful? Give feedback.
-
api key still needs to be removed from the comment, just in case you're still using the same key. |
Beta Was this translation helpful? Give feedback.
-
|
I have set all possible places where llm can be set to gemini-pro in the code, but when running the code, it still calls gpt4.0. Could you please help me identify the issue? The complete code is posted below. import os
import sys
from crewai import Agent, Task, Crew, Process
from langchain_google_genai import ChatGoogleGenerativeAI
class PromptImproveAgent:
def __init__(self):
default_llm = ChatGoogleGenerativeAI(
model="gemini-pro",
verbose=True,
temperature=0.1,
google_api_key=os.environ["GOOGLE_API_KEY"],
)
self.improver = Agent(
role="提示优化助理",
backstory="""你是一位资深的Gemini提示工程师，为用户提供优化的效果更好的ChatGPT提示语""",
goal="""帮助用户为用户的需求制定最佳Gemini提示语。""",
allow_delegation=False,
verbose=True,
llm=default_llm,
function_calling_llm=default_llm,
)
self.expect = Agent(
role="各领域专家",
goal="针对用户需求，提出专业的解决方案或答案",
verbose=True,
memory=True,
backstory="针对用户的要求，提出专业的解决方案，方案要精炼，不要进行太多解释。",
llm=default_llm,
function_calling_llm=default_llm,
)
self.improve_prompt_task = Task(
description=f"""优化用户提供的Gemini提示词{input}""",
agent=self.improver,
expected_output="修订提示（提供您重写的提示。它应该清晰、简洁，并且您可以轻松理解），没有其他任何描述和说明",
human_input=False,
)
self.answer_question_task = Task(
description=f"""根据优化过的提示词{input}，提供专业的解决方案，它应该清晰、简洁，不需要任何解释""",
agent=self.expect,
expected_output="解决方案，没有任何多余解释",
human_input=False,
)
self.crew = Crew(
agents=[self.improver, self.expect],
tasks=[self.improve_prompt_task, self.answer_question_task],
process=Process.sequential,
memory=True,
max_rpm=100,
share_crew=True,
manager_llm=default_llm,
function_calling_llm=default_llm,
)
def process(self, input: str):
result = self.crew.kickoff(inputs={"input": input})
return result
if __name__ == "__main__":
agent = PromptImproveAgent()
user_input = ""
# If called directly from the command line take the first argument as the filename
while user_input != "/exit":
user_input = input("请输入您的prompt: ")
result = agent.process("我的提示: " + user_input)
print(result) |
Beta Was this translation helpful? Give feedback.
-
|
Not sure what is the issue, but I copied the code you shared and it ran with me with Gemini.
Try to move the file to another folder structure or run it on Google Colab, to make sure it is not reading something locally or caching. hopefully, someone else could help. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you, I've tried on Google Colab, but it still called openai |
Beta Was this translation helpful? Give feedback.
-
|
I changed the crew config, it works now. self.crew = Crew(
agents=[self.improver, self.expect],
tasks=[self.improve_prompt_task, self.answer_question_task],
process=Process.sequential,
memory=False,
max_rpm=100,
share_crew=False,
manager_llm=default_llm,
function_calling_llm=default_llm,
) |
Beta Was this translation helpful? Give feedback.
-
|
When you set memory = True, it utilizes its internal module called entity_memory,short_term_memory also when you are using sequential process, you don't have to define manager_llm as it is only needed in hierarchical process. if you want to use the memory then you can use this code: |
Beta Was this translation helpful? Give feedback.
-
|
@punitchauhan771 Thank you very much! The explanation was very clear and detailed, I understand now. |
Beta Was this translation helpful? Give feedback.
-
Just to inform you, Gemini now supports function calling. It would be great if you could update the code to use the latest release version of the Google Gemini library. reference code: |
Beta Was this translation helpful? Give feedback.
-
|
Why isn't the model = "gemini-1.5-pro-latest" ? |
Beta Was this translation helpful? Give feedback.
-
|
I'm running it on Google Colab from crewai import Agent llm=ChatGoogleGenerativeAI(model="gemini-1.5-flash", ERROR: /usr/lib/python3.10/os.py in getitem(self, key) KeyError: 'OPENAI_API_KEY' |
Beta Was this translation helpful? Give feedback.
-
in your agent, you need to add llm argument because by default it used OpenAI model |
Beta Was this translation helpful? Give feedback.
-
|
I did add the llm argument, but I can still see the issue "Provider List: https://docs.litellm.ai/docs/providers 2024-10-04 13:06:32,803 - 8480485952 - llm.py-llm:104 - ERROR: Failed to get supported params: argument of type 'NoneType' is not iterable Provider List: https://docs.litellm.ai/docs/providers" please check this bug : #1393 |
Beta Was this translation helpful? Give feedback.
-
|
@punitchauhan771 @Iris-Liu96 Setting up hugging face tokenos.environ["HUGGINGFACE_ACCESS_TOKEN"] = "hf_..."And then accessing embedding model from huggingface in instantiating the crewcrew = Crew(
agents=[..],
...,
...,
embedder=dict(
provider="huggingface", # or openai, ollama, ...
config=dict(
model="mixedbread-ai/mxbai-embed-large-v1",
),
),
)I tried But I'm still not sure tbh if this embeddor/ memory = True actually does work/help in anything. The output with memory=True barely differs from otherwise not including memory. The search/serper calls seem to be same, even after final thought, crew seems to go back to beginning and start again (I'm following the getting started guide) Quick question - I am practically trying to do everything free of cost and using frameworks has been a nightmare because of there default use of OpenAI, should I completely ditch these frameworks and go to basic langchain and langgraph to create agent workflows from ground up or try my hands at different frameworks (if there are any that allow high degree of customizability without having to read multiple docs please educate me). Becauase langchain and langgraph are so low level when it comes to ai agents, I'm a bit hesitant to put in that much work and complexity yk |
Beta Was this translation helpful? Give feedback.
-
|
Hi @AdityaPrasad275 , |
Beta Was this translation helpful? Give feedback.
-
|
Google Gemini already supported |
Beta Was this translation helpful? Give feedback.
-
Since Google Gemini Pro API is currently free when using up to 60 API calls per minute. This would be an incredibly helpful integration to add support for Gemini API into the CrewAI code. This will perform better than GPT 3.5 without any API fees. However, CrewAI must be able to simply allow users to define an API call/Functions Limit, so they can set it to 59 API processes per minute to avoid going over the free tier use per minute limits.
Beta Was this translation helpful? Give feedback.
All reactions