Add cohere chat generator #88

sunilkumardash9 · 2023-12-07T21:09:31Z

support for cohere chat endpoint

Signed-off-by: sunilkumardash9 <[email protected]>

CLAassistant · 2023-12-07T21:09:38Z

All committers have signed the CLA.

vblagoje · 2023-12-08T08:20:06Z

Thanks for this effort @sunilkumardash9 Can you use the existing ChatMessage like other layers do?

Signed-off-by: sunilkumardash9 <[email protected]>

update repo

Signed-off-by: sunilkumardash9 <[email protected]>

sunilkumardash9 · 2023-12-08T17:11:06Z

Hi @vblagoje, I was curious how this import works as I can't seem to import chat_generator to test files with relative import.

haystack-core-integrations/integrations/cohere/tests/test_document_embedder.py

Line 10 in 4076949

from cohere_haystack.embedders.document_embedder import CohereDocumentEmbedder

Signed-off-by: sunilkumardash9 <[email protected]>

vblagoje · 2023-12-14T14:15:15Z

integrations/cohere/src/cohere_haystack/chat/chat_generator.py

+                    stream_chunk = self._build_chunk(chunk)
+                    self.streaming_callback(stream_chunk)
+            chat_message = ChatMessage(content=response.texts, role=None, name=None)
+            chat_message.metadata.update(


Any way we can try to match OpenAI metadata, they have:

{ "model": chunk.model, "index": 0, "finish_reason": "stop" "usage": {}, # here we put token counts for prompt and response }

Aha I see token counts are available. @anakin87 should we try to match the OpenAI format here so all the chat generators are more or less interchangeable? Ideally we match the OpenAI format and then provide more Cohere specific metadata, whatever is available, can't hurt

It would be nice to do that!

vblagoje

@sunilkumardash9 solid work, thank you. Would you please address my minor comments and don't forget to add release note using reno tool. See contributing section, thanks for your great work so far.

vblagoje · 2023-12-14T14:18:02Z

integrations/cohere/src/cohere_haystack/chat/chat_generator.py

+                    stream_chunk = self._build_chunk(chunk)
+                    self.streaming_callback(stream_chunk)
+            chat_message = ChatMessage(content=response.texts, role=None, name=None)
+            chat_message.metadata.update(


Aha I see token counts are available. @anakin87 should we try to match the OpenAI format here so all the chat generators are more or less interchangeable? Ideally we match the OpenAI format and then provide more Cohere specific metadata, whatever is available, can't hurt

2. add doc strings 3. change metadata to match other chat generators Signed-off-by: sunilkumardash9 <[email protected]>

update repo

vblagoje · 2023-12-20T10:25:12Z

Apologies for the delay @sunilkumardash9 I'll take a look on Friday, play with the generator and hopefully, we will integrate it at that time.

sunilkumardash9 · 2023-12-21T13:52:41Z

@vblagoje No issues. Do let me know in case I have missed anything.

vblagoje · 2023-12-22T09:56:42Z

Hey @sunilkumardash9 if I understood their chat API correctly, I think we need to set the message parameter to be our last message, i.e message[-1], not to concatenate all messages into one and then sending it over. Rather, we need to pass all messages before the last as chat_history parameter. But please correct me if I'm wrong, I'm looking at the API docs; you have played with the model.

Also please consider playing with an example chat script to prove to yourself that everything is working as expected after these changes.

vblagoje · 2023-12-22T10:01:45Z

Also it seems like we have to translate the standard roles that everyone else agreed upon to cohere roles "User", and "Chatbot". So I think you'll need to do a bit of conversion of ChatMessages given to the CohereChatGenerator run method.

user_message = {"user_name": "User", "text": message}
bot_message = {"user_name": "Chatbot", "text": answer}

2. Adds a unit test for 1 Signed-off-by: sunilkumardash9 <[email protected]>

sunilkumardash9 · 2023-12-23T09:41:20Z

@vblagoje I was wondering the same after Gemini chat, which has a similar chat history implementation, made some changes to adjust default chat roles to Cohere-specific ones. And it seems to be working well.

vblagoje · 2023-12-23T15:29:53Z

Excellent @sunilkumardash9 took a quick look and this is much better. I'll play with it over the holidays and get back to you early next week

vblagoje · 2023-12-27T10:44:08Z

Hey @sunilkumardash9 , I tried your CohereChatGenerator, and it works well. The only issue I faced was in streaming mode. The first response from LLM streams fine, but when I tried continuing the chat conversation, the second request failed with 400 response. Out of curiosity - do you get the same issue?

sunilkumardash9 · 2023-12-27T11:16:44Z

@vblagoje It is working fine for me, even with subsequent requests. There might be some issues with the Cohere endpoint.

vblagoje · 2023-12-27T13:57:39Z

@sunilkumardash9, this looks okay to me as the first version. Let's track the progress at #150 . Please update the task completion there so we can push it toward the finish line.

vblagoje

Thanks for this contribution @sunilkumardash9

sunilkumardash9 · 2023-12-27T15:32:26Z

@vblagoje I think most tasks in #150 have already been done while CohereGenerator was added. Would you like me to complete the rest of it?

anakin87 · 2023-12-27T16:50:37Z

Sorry sorry...

For consistency, we are changing each mention of metadata in the codebase to meta: see deepset-ai/haystack#6570

@sunilkumardash9 can you please update this PR to reflect this change?
Otherwise, I will take care...

vblagoje · 2023-12-27T20:16:11Z

Right, I've seen it too and forgot to mention it to @sunilkumardash9 - thanks @anakin87 Please update these references @sunilkumardash9

update repo

sunilkumardash9 · 2023-12-28T12:17:12Z

Hi @anakin87, I am getting metadata references in chat_message and chat_role in haystack-ai v2.0.0b3.

anakin87 · 2023-12-28T12:21:07Z

@sunilkumardash9 Sorry, my fault.
We haven't released the new version yet, so it is OK to merge this PR.
When we release the new version of Haystack (with modified ChatMessages), we will also update your implementation.

anakin87 · 2023-12-28T12:22:10Z

integrations/cohere/releasenotes/notes/add_cohere_chat_generator-ef96bbeaf588281c.yaml

+---
+preview:
+  - |
+    Adds `CohereChatGenerator` for text and chat generation.


We currently don't have release notes for this repository.
Please remove this file.

Signed-off-by: sunilkumardash9 <[email protected]>

add cohere chat generator

837261d

Signed-off-by: sunilkumardash9 <[email protected]>

sunilkumardash9 requested a review from a team as a code owner December 7, 2023 21:09

sunilkumardash9 requested review from vblagoje and removed request for a team December 7, 2023 21:09

sunilkumardash9 changed the title ~~add cohere chat generator~~ WIP: add cohere chat generator Dec 7, 2023

sunilkumardash9 and others added 3 commits December 8, 2023 14:06

remove chat_message.py

f9840ad

Signed-off-by: sunilkumardash9 <[email protected]>

Merge pull request #1 from deepset-ai/main

d56b347

update repo

add unit and integration tests

ed81a0d

Signed-off-by: sunilkumardash9 <[email protected]>

sunilkumardash9 added 3 commits December 10, 2023 18:33

improve tests

4f6432b

Signed-off-by: sunilkumardash9 <[email protected]>

fix lint errors

d2e08e8

Signed-off-by: sunilkumardash9 <[email protected]>

fix lint errors(1)

f97ccd1

Signed-off-by: sunilkumardash9 <[email protected]>

sunilkumardash9 changed the title ~~WIP: add cohere chat generator~~ Add cohere chat generator Dec 11, 2023

masci added 2 commits December 11, 2023 19:21

Merge branch 'main' into main

4725c48

Merge branch 'main' into main

f14a046

github-actions bot added the integration:cohere label Dec 12, 2023

vblagoje reviewed Dec 14, 2023

View reviewed changes

vblagoje requested changes Dec 14, 2023

View reviewed changes

sunilkumardash9 and others added 3 commits December 16, 2023 02:15

1. add releasenote

bc9674c

2. add doc strings 3. change metadata to match other chat generators Signed-off-by: sunilkumardash9 <[email protected]>

Merge pull request #2 from deepset-ai/main

ea574cc

update repo

Merge remote-tracking branch 'origin/main'

33f218e

masci assigned vblagoje Dec 18, 2023

1. Adds ChatRole and convert default role to Cohere compliant role

518a6b9

2. Adds a unit test for 1 Signed-off-by: sunilkumardash9 <[email protected]>

vblagoje self-requested a review December 27, 2023 13:57

vblagoje approved these changes Dec 27, 2023

View reviewed changes

Merge pull request #3 from deepset-ai/main

ea4c027

update repo

anakin87 reviewed Dec 28, 2023

View reviewed changes

remove releasenotes

3d3c924

Signed-off-by: sunilkumardash9 <[email protected]>

vblagoje merged commit 492a0ae into deepset-ai:main Dec 28, 2023
9 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add cohere chat generator #88

Add cohere chat generator #88

sunilkumardash9 commented Dec 7, 2023 •

edited

Loading

CLAassistant commented Dec 7, 2023 •

edited

Loading

vblagoje commented Dec 8, 2023

sunilkumardash9 commented Dec 8, 2023

vblagoje Dec 14, 2023

vblagoje Dec 14, 2023

anakin87 Dec 14, 2023

vblagoje left a comment

vblagoje Dec 14, 2023

vblagoje commented Dec 20, 2023

sunilkumardash9 commented Dec 21, 2023

vblagoje commented Dec 22, 2023 •

edited

Loading

vblagoje commented Dec 22, 2023

sunilkumardash9 commented Dec 23, 2023

vblagoje commented Dec 23, 2023

vblagoje commented Dec 27, 2023

sunilkumardash9 commented Dec 27, 2023

vblagoje commented Dec 27, 2023 •

edited

Loading

vblagoje left a comment

sunilkumardash9 commented Dec 27, 2023

anakin87 commented Dec 27, 2023

vblagoje commented Dec 27, 2023

sunilkumardash9 commented Dec 28, 2023

anakin87 commented Dec 28, 2023

anakin87 Dec 28, 2023

Add cohere chat generator #88

Add cohere chat generator #88

Conversation

sunilkumardash9 commented Dec 7, 2023 • edited Loading

CLAassistant commented Dec 7, 2023 • edited Loading

vblagoje commented Dec 8, 2023

sunilkumardash9 commented Dec 8, 2023

vblagoje Dec 14, 2023

Choose a reason for hiding this comment

vblagoje Dec 14, 2023

Choose a reason for hiding this comment

anakin87 Dec 14, 2023

Choose a reason for hiding this comment

vblagoje left a comment

Choose a reason for hiding this comment

vblagoje Dec 14, 2023

Choose a reason for hiding this comment

vblagoje commented Dec 20, 2023

sunilkumardash9 commented Dec 21, 2023

vblagoje commented Dec 22, 2023 • edited Loading

vblagoje commented Dec 22, 2023

sunilkumardash9 commented Dec 23, 2023

vblagoje commented Dec 23, 2023

vblagoje commented Dec 27, 2023

sunilkumardash9 commented Dec 27, 2023

vblagoje commented Dec 27, 2023 • edited Loading

vblagoje left a comment

Choose a reason for hiding this comment

sunilkumardash9 commented Dec 27, 2023

anakin87 commented Dec 27, 2023

vblagoje commented Dec 27, 2023

sunilkumardash9 commented Dec 28, 2023

anakin87 commented Dec 28, 2023

anakin87 Dec 28, 2023

Choose a reason for hiding this comment

sunilkumardash9 commented Dec 7, 2023 •

edited

Loading

CLAassistant commented Dec 7, 2023 •

edited

Loading

vblagoje commented Dec 22, 2023 •

edited

Loading

vblagoje commented Dec 27, 2023 •

edited

Loading