Feat/add nvidia nim

[gurdeep330]

For authors

Description

demo11.mp4

In this PR, I'd like to add open-source text-generation and text-to-embedding models by NVIDIA NIM to T2B.

Changes Introduced

1. Added a new key (text_embedding_model) in the state to store text-to-embedding models (type Embeddings class in LangGraph).

2. Updated the existing key (llm_model) in the state to store chat model objects (type BaseChatModel in LangGraph).

   • By tweaking just the base_url parameter in the ChatNVIDIA object, users can switch from NVIDIA API to local deployment as described here.

3. Updated the Streamlit app to provide users with LLM selection options:

   • gpt-40-mini (OpenAI/Default)
   • meta/llama-3.3-70b-instruct (NVIDIA NIM)

   and Embedding model options:

   • nvidia/llama-3.2-nv-embedqa-1b-v2 (NVIDIA NIM/Default)
   • text-embedding-ada-002 (OpenAI)

4. A small majority of tests now use NVIDIA NIM models.

5. Future Enhancements:

   • Plan to implement the NVIDIA Blueprint for RAG

Fixes # (issue) N/A

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

Please describe the tests you conducted to verify your changes. These may involve creating new test scripts or updating existing ones.

• Added new test(s) in the tests folder
• Added new function(s) to an existing test(s) (e.g.: tests/testX.py)
• No new tests added (Please explain the rationale in this case)

Checklist

• My code follows the style guidelines mentioned in the Code/DevOps guides
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation (e.g. MkDocs)
• My changes generate no new warnings
• I have added or updated tests (in the tests folder) that prove my fix is effective or that my feature works
• New and existing tests pass locally with my changes
• Any dependent changes have been merged and published in downstream modules

For reviewers

Checklist pre-approval

• Is there enough documentation?
• If a new feature has been added, or a bug fixed, has a test been added to confirm good behavior?
• Does the test(s) successfully test edge/corner cases?
• Does the PR pass the tests? (if the repository has continuous integration)

Checklist post-approval

• Does this PR merge develop into main? If so, please make sure to add a prefix (feat/fix/chore) and/or a suffix BREAKING CHANGE (if it's a major release) to your commit message.
• Does this PR close an issue? If so, please make sure to descriptively close this issue when the PR is merged.

Checklist post-merge

• When you approve of the PR, merge and close it (Read this article to know about different merge methods on GitHub)
• Did this PR merge develop into main and is it suppose to run an automated release workflow (if applicable)? If so, please make sure to check under the "Actions" tab to see if the workflow has been initiated, and return later to verify that it has completed successfully.

[dmccloskey]

The integration for NVIDIA NIMs looks really cool 😎. The change in url is a neat trick to switch between API and local too.

I only had small comment on the logic implemented to switch between models on the front end that I think will also allow for better readability of the display names. Let me know your thoughts if you disagree.

[gurdeep330]

@dmccloskey I have implemented your suggestion.

While testing the application, I noticed that Streaming with ChatNVIDIA does not work in tool-calling. I see that others have also opened issues regarding this on GitHub. So for now, I have disabled streaming when using ChatNVIDIA (only invoke possible). I will get back to this when I have found a reliable solution.

Until then streaming is enabled only for OpenAI models.

[dmccloskey]

That is unfortunate about NVIDIA streaming 🙁. Hopefully, they will fix the issue quickly.

[github-actions]

🎉 This PR is included in version 1.18.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀