Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: AmazonBedrockChatGenerator - add tools support #1304

Merged
merged 17 commits into from
Jan 23, 2025
Merged

Conversation

vblagoje
Copy link
Member

@vblagoje vblagoje commented Jan 20, 2025

Why:

Adds Haystack tool support to AmazonBedrockChatGenerator

What:

  • Added method to convert Haystack tools to Amazon Bedrock toolConfig format, facilitating tool integration.
  • Modified AmazonBedrockChatGenerator to accept, serialize, and handle tool specifications as part of the chat messages.
  • Updated testing suite to cover tool-related functionalities, ensuring robustness and reliability.

How can it be used:

The changes enable users to pass tools to AmazonBedrockChatGenerator, enhancing the model's capabilities by allowing it to use external tools within chat interactions. Here is a simplified example:

from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.amazon_bedrock import AmazonBedrockChatGenerator
from haystack.tools import Tool

def weather(city: str):
        """Get weather for a given city."""
        return f"The weather in {city} is sunny and 32°C"

tool_parameters = {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}
tool = Tool(
    name="weather",
    description="useful to determine the weather in a given location",
    parameters=tool_parameters,
    function=weather,
)

messages = [ChatMessage.from_user("What's the weather like in Paris?")]
generator = AmazonBedrockChatGenerator(model="model_name", tools=[tool])
response = generator.run(messages=messages)

How did you test it:

Implemented thorough unit and integration tests to validate the addition of tool-related functionalities. Tests included:

  • Serialization and deserialization of tool configurations.
  • Ensuring tool calls within conversations behave as expected.
  • Integration tests with mock and live scenarios for tool usage within chat interactions.

Notes for the reviewer:

Please pay special attention to the serialization and deserialization processes for tool objects and the handling of tool calls within the chat messages. Consider reviewing the edge cases in the test suite that handle different orderings of tool messages in replies.

@github-actions github-actions bot added integration:amazon-bedrock type:documentation Improvements or additions to documentation labels Jan 20, 2025
@vblagoje vblagoje marked this pull request as ready for review January 22, 2025 11:01
@vblagoje vblagoje requested a review from a team as a code owner January 22, 2025 11:01
@vblagoje vblagoje requested review from julian-risch and anakin87 and removed request for a team January 22, 2025 11:01
@vblagoje vblagoje marked this pull request as draft January 22, 2025 14:30
@vblagoje vblagoje marked this pull request as ready for review January 22, 2025 15:29
Copy link
Member

@anakin87 anakin87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have some questions and small suggestions.

In general, this PR looks good.

)

@pytest.mark.parametrize("model_name", [MODELS_TO_TEST_WITH_TOOLS[0]]) # just one model is enough
@pytest.mark.integration
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would transform this into a simpler unit test about serde (without running the pipeline).
Pipeline running with tools is already tested in test_pipeline_with_amazon_bedrock_chat_generator.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please simplify this test

Copy link
Member Author

@vblagoje vblagoje left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah missing the pydoc class tooling example. I'll add it @anakin87

@vblagoje vblagoje requested a review from anakin87 January 23, 2025 15:22
@vblagoje
Copy link
Member Author

@anakin87 lmk what we should do about ids or perhaps we can do that in separate PR? I'm ok with both

Copy link
Member

@anakin87 anakin87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At this point, let's do nothing about ID and wait for @lambda-science's opinion in the issue.

Please take this comment into account: #1304 (comment)
Then this PR is ready to be merged.

@vblagoje
Copy link
Member Author

At this point, let's do nothing about ID and wait for @lambda-science's opinion in the issue.

Please take this comment into account: #1304 (comment) Then this PR is ready to be merged.

Right - I forgot about that one 🙏
Will correct now

@vblagoje
Copy link
Member Author

Ok should be gtg now @anakin87 See 91f2330

@vblagoje vblagoje requested a review from anakin87 January 23, 2025 15:48
Copy link
Member

@anakin87 anakin87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

last change request (hopefully 😊)

Comment on lines 577 to 578
@pytest.mark.parametrize("model_name", [MODELS_TO_TEST_WITH_TOOLS[0]]) # just one model is enough
@pytest.mark.integration
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
@pytest.mark.parametrize("model_name", [MODELS_TO_TEST_WITH_TOOLS[0]]) # just one model is enough
@pytest.mark.integration
  • let's transform this into a unit test (with mocking?)
  • if possible, let's test the dictionary format more extensively
  • let's also dump to YAML and reload from YAML (some serialization errors happen in this phase)

An example of what I mean:

@vblagoje
Copy link
Member Author

Ok, I agree, how about now? 🤞

@vblagoje vblagoje requested a review from anakin87 January 23, 2025 16:05
Copy link
Member

@anakin87 anakin87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@vblagoje vblagoje merged commit 3d0dfed into main Jan 23, 2025
11 checks passed
@vblagoje vblagoje deleted the bedrock_tools branch January 23, 2025 16:10
Amnah199 pushed a commit that referenced this pull request Jan 28, 2025
* AmazonBedrockChatGenerator - add tools support

* Remove test not needed

* Add actual pipeline integration test with tools

* Extract instance functions to free standing

* No need to test serde on all models

* Add serde test

* Fix serde test

* Lint

* Always pack thinking + tool call into single ChatMessage

* Revert accidental changes

* Method renaming for Python Zen

* Add class pydocs

* Don't run pipeline in serde test

* Update test_serde_in_pipeline test

* Lint
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bedrock ChatGenerator - support for Tool
2 participants