[Bug]: Gemini doesn't support function calling

[vcoolish]

Describe the bug

Both google api type and using gemini via openai api type doesn't help. Agent is not capable of calling functions.

Steps to reproduce

No response

Model Used

gemini 2.0 flash

Expected Behavior

No response

Screenshots and logs

No response

Additional Information

v0.7.5

[marklysze]

Hy @vcoolish, thanks for raising this, can please provide some sample code to test? I was able to get it to execute tools today with the api_type = "google".

[vcoolish]

yes sure

config_list = [
    {
        "model": "gemini-2.0-flash",
        "api_key": os.getenv("GOOGLE_API_KEY", ""),
        "api_type": "google",
    }
]
llm_config_proxy = {
    "timeout": 600,
    "cache_seed": None,
    "config_list": config_list,
}
llm_config_assistant = {
    "timeout": 600,
    "cache_seed": None,
    "config_list": config_list,
    "functions": [
        {
            "name": "web_search",
            "description": "Perform a web search using Brave Search API and return results.",
            "parameters": {
                "type": "object",
                "properties": {
                    "query": {"type": "string", "description": "Web search query"}
                },
                "required": ["query"],
            },
        },
    ],
}

def web_search(query: str) -> str:
    """
    Perform a web search using Brave Search API and return results.
    """
    brave_api_key = os.getenv("BRAVE_API_KEY")
    if not brave_api_key:
        return {
            "error": "Brave API key is missing. Please set BRAVE_API_KEY in your .env file."
        }

    search_url = "https://api.search.brave.com/res/v1/web/search"
    headers = {"X-Subscription-Token": brave_api_key}
    params = {"q": query, "count": 5}

    try:
        response = requests.get(search_url, headers=headers, params=params)
        response.raise_for_status()
        results = response.json()

        search_results = [
            {
                "title": item.get("title"),
                "url": item.get("url"),
                "snippet": item.get("description"),
            }
            for item in results.get("web", {}).get("results", [])
        ]
        return json.dumps(search_results)

    except requests.exceptions.RequestException as e:
        return {"error": f"Web search failed: {str(e)}"}

class AutogenChat:
    def __init__(self, chat_id=None, websocket=None):
        self.websocket = websocket
        self.chat_id = chat_id
        self.client_sent_queue = Flow()
        self.client_receive_queue = Flow()
        self.turn_count = 0
        self.enc = tiktoken.encoding_for_model("gpt-4o-mini")

        with open("system_prompt.txt", "r") as file:
            system_message = file.read()

        self.assistant = autogen.AssistantAgent(
            name="assistant",
            llm_config=llm_config_assistant,
            system_message=system_message,
        )
        self.user_proxy = UserProxyWebAgent(
            name="user_proxy",
            human_input_mode="ALWAYS",
            max_consecutive_auto_reply=10,
            is_termination_msg=lambda x: isinstance(x.get("content"), str)
            and (
                x.get("content").rstrip().endswith("TERMINATE")
                or "summary" in x.get("content")
            ),
            code_execution_config=False,
            function_map={
                "web_search": web_search,
            },
        )

also tried this

autogen.register_function(
            web_search,
            caller=self.assistant,
            executor=self.user_proxy,
            name="web_search",
            description="Searches web with brave API",
        )

and this function decorators too

[vcoolish]

here is the log

user_proxy (to assistant):

search for current market trends on coinmarketcap

--------------------------------------------------------------------------------
/Users/vcoolish/Documents/AIProjects/twwssample/venv/lib/python3.12/site-packages/autogen/oai/gemini.py:850: UserWarning: Cost calculation is not implemented for model gemini-2.0-flash. Cost will be calculated zero.
  warnings.warn(
assistant (to user_proxy):

```json
{
  "action": "info",
  "summary": "I am unable to directly access CoinMarketCap or perform market analysis. However, I can search the web for general information on crypto market trends. Would you like me to do that?"
}

---

user_proxy (to assistant):

yes

---

assistant (to user_proxy):

{
  "action": "info",
  "summary": "Okay, I will perform a web search for general information on crypto market trends."
}

---

nothing happens after that.