feat: Add AzureOpenAIGenerator and AzureOpenAIChatGenerator #6648
Conversation
vblagoje
requested review from
dfokina and
julian-risch
and removed request for
a team
github-actions
bot
added
2.x
|
@vblagoje I saw that there are two issues about this topic here: #6620 and here: deepset-ai/haystack-core-integrations#141 What do you think about those? Why not put this PR into the haystack-core-integrations? |
|
Alright, after a quick sync we decided that AzureGenerator can stay in core for now. It's very similar to OpenAIGenerator using just a different client and it doesn't add any dependencies. If a good reason to move it to core-integrations comes up later, we can do that. |
There was a problem hiding this comment.
Looks very good to me already. I just have a few smaller change requests.
- Skipping the super. _ _init _ _ call is fine. Adding a comment there would help readability.
- Let's rename to
AzureOpenAIGeneratorandAzureOpenAIChatGenerator. - We should add unit tests at least for
to_dict,from_dictandinitas these methods are not covered by the OpenAIGenerator tests.
| Bigger values mean the model will be less likely to repeat the same token in the text. | ||
| - `logit_bias`: Add a logit bias to specific tokens. The keys of the dictionary are tokens, and the | ||
| values are the bias to add to that token. | ||
| """ |
There was a problem hiding this comment.
It can be helpful to add a comment here explaining that super().init() call is skipped on purpose and that this is done because we need to initialize an AzureOpenAI client instead of the OpenAI client.
vblagoje
force-pushed
the
azure_generators
branch
from
32a528f to
ed7538a
Compare
|
Ready for another review @julian-risch 🚀 |
julian-risch
changed the title
| --- | ||
| features: | ||
| - | | ||
| Adds support for Azure OpenAI models with AzureGenerator and AzureChatGenerator components. |
There was a problem hiding this comment.
Let's rename here to AzureOpenAIGenerator and AzureOpenAIChatGenerator
Why:
Add support for OpenAI LLMs like GPT-3.5 and GPT-4, which are available through Azure's services.
What:
Two main components have been added:
AzureGenerator: This class extends the capabilities of the
OpenAIGeneratorto use Azure's deployment of OpenAI's models. It involves numerous parameters and methods that facilitate text generation with customizable settings tailored to Azure's API structure and authentication mechanisms.AzureChatGenerator: Similar to
AzureGenerator, this class is tailored for chat-based applications, providing functionalities to utilize Azure's deployment of OpenAI models in chatbots or other conversational AI applications.Additionally, both components are registered in their respective
__init__.pyfiles.How can it be used:
Developers and researchers can use these components to integrate Azure's powerful language models into their applications for a variety of tasks, including but not limited to:
These generators provide easy access to advanced NLP capabilities and can be customized with different parameters to suit specific needs, such as adjusting the model's creativity or response length.
How did you test it:
Manually. Unit tests were not added as we use the already tested OpenAIGenerator and OpenAIChatGenerator
Notes for the reviewer:
Omission of super init call: Azure generators intentionally omit super init call to parent components. This is intentional, see if that is the best approach possible.
Understanding Azure: As these changes involve integration with Azure, understanding the Azure environment, its authentication mechanisms, and how it hosts OpenAI models is crucial.
Reviewing Dependencies: Changes include imports specific to Azure's SDK (
openai.lib.azure) and other related utilities, which should be reviewed for compatibility and stability.Considerations for Deployment: Ensure that the addition of these new components aligns with the overall architecture and does not introduce any issues or conflicts with existing functionalities.