Adding support for ⚡ Groq API

[someshfengde]

Now we can use Groq API within dspy.

Makes querying to LLMs lot faster. 💨

Thanks @ivarflakstad for helping me debug.

Usage:

import dspy
groq_model = dspy.GROQ(api_key ="gsk_***" )
dspy.settings.configure(lm = groq_model)
class BasicQA(dspy.Signature):
     "hi mom"
     question = dspy.InputField()
     answer = dspy.OutputField(desc="often between 1 and 5 words")
 
generate_answer = dspy.Predict(BasicQA)
pred = generate_answer(question="what is the meaning of life")
print(pred)

Output

Prediction(
    answer='Meaning varies person to person.'
)

[insop]

@ivarflakstad thank you for the PR.
Could you add the documentation at docs/api/language_model_clients

[ivarflakstad]

@someshfengde deserves the thanks😊

[someshfengde]

yes sure @insop will do it. :)

[someshfengde]

Hi @insop I've added the documentation for the same :)

[insop]

Thank you @someshfengde , looks good to me.

[someshfengde]

@insop removed comment also resolved merge conflicts

[ebudmada]

Very great, can't handle myself the see the mistral8x7b tests. Groq is very cheap, can it be 0,27$ for million tokens?! Great great great ! Thank you @someshfengde

[pfa34488]

When I run the script. I get dspy cannot find GROQ. I pip install dspy-ai. Am I missing anything?

[ebudmada]

@pfa34488 Yes you have to integrate the changes yourself! This workflow is waiting approval

[pfa34488]

Thanks so much! I see the module at curieo.org/dspy under dsp modules; I see from .groq_client import * and the groq_client.py file. I am not sure how to add. Do i find the pip install dspy-ai or git clone (https://github.com/curieo-org/dspy.git) or something else? Please help!

[ebudmada]

Yes there is 8 file change in this PR, do it yourself or wait until it is implemented

[okhat]

Thanks so much. Checks fail because this has imports that are always on but are only needed for those that use this class. Please see how cohere and many other modules do this, for a different pattern: import on demand.

[pfa34488]

I got it LoL. You guys are sharp. From Newbe

[someshfengde]

Hi @pfa34488 you can clone a branch from repo with branch initial-groq-support.

Sorry currently you can't work with the main branch I'm resolving an error for The Together api over main.

[someshfengde]

Very great, can't handle myself the see the mistral8x7b tests. Groq is very cheap, can it be 0,27$ for million tokens?! Great great great ! Thank you @someshfengde

Welcome 🤗

[someshfengde]

When I run the script. I get dspy cannot find GROQ. I pip install dspy-ai. Am I missing anything?

It's still not integrated into the main branch. It'll soon be integrated 🙂

[someshfengde]

Thanks so much. Checks fail because this has imports that are always on but are only needed for those that use this class. Please see how cohere and many other modules do this, for a different pattern: import on demand.

Yes sure will do it 👍🏻

[arnavsinghvi11]

Hi @someshfengde , the CI tests seem to still fail. Could you run these following commands to fix it?

Run Tests failure:
Warning: poetry.lock is not consistent with pyproject.toml. You may be getting improper dependencies. Run poetry lock [--no-update] to fix it.

Check Ruff Fix (formatting) failure:
ruff check . --fix-only

Build Poetry failure (You will have to make changes to fix the error triggered below):

Run python -c "import dspy"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/runner/work/dspy/dspy/dspy/__init__.py", line 1, in <module>
    import dsp
  File "/home/runner/work/dspy/dspy/dsp/__init__.py", line 1, in <module>
    from .modules import *
  File "/home/runner/work/dspy/dspy/dsp/modules/__init__.py", line [12](https://github.com/stanfordnlp/dspy/actions/runs/8431136781/job/23113260978?pr=637#step:8:13), in <module>
    from .together_client import *
  File "/home/runner/work/dspy/dspy/dsp/modules/together_client.py", line 4, in <module>
    from typing import Any, Literal, Optional, Required
ImportError: cannot import name 'Required' from 'typing' (/opt/hostedtoolcache/Python/3.9.[18](https://github.com/stanfordnlp/dspy/actions/runs/8431136781/job/23113260978?pr=637#step:8:19)/x64/lib/python3.9/typing.py)
Error: Process completed with exit code 1.

[arnavsinghvi11]

I've also left another comment for removing the extraneous files from .gitignore

[arnavsinghvi11]

Also, can you remove all the related Together changes? seems they have been added in some extra commits I didn't see before but they are unrelated to the Groq LM in this PR and are modifying the integrated dspy.Together LM. Let me know if you have any questions about this (but these changes could possibly be impacting the test failures too!)

[someshfengde]

Hi @arnavsinghvi11 Removed the together related files, removed the gitignore added files too. Updated the poetry lock with poetry lock --no-update
and ran ruff check . --fix-only too

Thanks for the review

[arnavsinghvi11]

Hi @someshfengde , the tests pass but there are still conflicts. Can you check if its a merge conflict on your end?

if not, I will checkout and merge the changes to main through command line.

[someshfengde]

Hi @someshfengde , the tests pass but there are still conflicts. Can you check if its a merge conflict on your end?

if not, I will checkout and merge the changes to main through command line.

@arnavsinghvi11
it was poetry lock conflict for hash I resolved it for now . if someone push new commit that hash is different causing conflict in PR

[okhat]

Happy to merge here once conflicts are gone

[someshfengde]

Hi @okhat for every other commit in poetry lock file will cause the merge conflict I've been resolving these conflicts lot's of times already 😢

[okhat]

Ah got it. Why not just remove/revert the lock and other files you don’t need to change? You just need to change like 4-5 files

[someshfengde]

Ah got it. Why not just remove/revert the lock and other files you don’t need to change? You just need to change like 4-5 files

yes just a sec will do it rn

[someshfengde]

@okhat it's done

[arnavsinghvi11]

Hi @someshfengde , it's still failing the poetry tests.

Can you run poetry lock --no-update and poetry install --no-interaction again?

[someshfengde]

Hi @someshfengde , it's still failing the poetry tests.

Can you run poetry lock --no-update and poetry install --no-interaction again?

I've ran this many times we are falling behind since every new push to the main repo will cause merge issues in this one?

[arnavsinghvi11]

After merging and running the commands, are there any updates you need to commit? seems like that should fix it or removing the poetry.lock which has extraneous changes.

[someshfengde]

@arnavsinghvi11 I've fixed the poetry lock for now

[arnavsinghvi11]

Closing since #773 is merged. Thanks again @someshfengde !