Gemini integration does not work with AnswerBuilder

[medsriha]

Describe the bug

I encountered the following error when connecting GoogleAIGeminiGenerator with the AnswerBuilder component.

haystack.core.errors.PipelineConnectError: Cannot connect 'llm.replies' with 'answer_builder.replies': their declared input and output types do not match.
'llm':
 - replies: List[Union[str, Dict[str, str]]]
'answer_builder':
 - replies: Union[List[str], List[ChatMessage]] (available)

To Reproduce

    from haystack import Pipeline
    from haystack.components.builders import AnswerBuilder
    from haystack_integrations.components.generators.google_ai import GoogleAIGeminiGenerator
    from haystack.utils import Secret

    gemini = GoogleAIGeminiGenerator(model="gemini-pro", api_key=Secret.from_env_var("GEMINI-API-KEY") )

    pipeline = Pipeline()

    pipeline.add_component(name="llm", instance=gemini)
    pipeline.add_component(name="answer_builder", instance=AnswerBuilder())

    pipeline.connect("llm.replies", "answer_builder.replies")

    res = pipeline.run(data={"llm": {"parts": "Hello"}, "answer_builder": {"query": "hello"}})

Describe your environment (please complete the following information):

• OS: [e.g. iOS] iOS
• Haystack version: v2.3.1
• Integration version:

[lbux]

This has to do with the way that the Google Generators are handling function calling (this also affects vertex). Other generators simply return a string with the message and the meta (which might contain the function data).

If the output type is changed to List[str], the error will go away, but we lose out on the function call return data. It can be added to meta if we also add meta as an output type.

I think there are three options:

• Remove the function call part and redirect users to the Chat versions which can better handle functions through the use of roles
• Store the function return in the meta and leave it to the user to extract it as they please.
• Temporarily remove function calling until haystack-experimental's version of Tool usage is brought to main

[acompa]

Checking in as another affected party here. Thanks for investigating!

[anakin87]

I'm working on this: the idea is to remove tools support in the Generator; tools will be only supported in the corresponding Chat Generator.

In the meantime, one can use an OutputAdapter` to overcome this issue.