RankLLMRerank usage throws an error when used GPT (not only) when rank-llm version is > 0.12.8

[tymzar]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Soluton

Issue is resolved with the PR: #29154

Example Code

Code from the tutorial and rank-llm installed with version > 0.12.8:

from langchain.retrievers.contextual_compression import ContextualCompressionRetriever
from langchain_community.document_compressors.rankllm_rerank import RankLLMRerank

compressor = RankLLMRerank(top_n=3, model="gpt", gpt_model="gpt-3.5-turbo")
compression_retriever = ContextualCompressionRetriever(
    base_compressor=compressor, base_retriever=retriever
)

Error Message and Stack Trace

---

ModuleNotFoundError Traceback (most recent call last)
File ~/Library/Caches/pypoetry/virtualenvs/nlp-rag-NMztfdt9-py3.10/lib/python3.10/site-packages/langchain_community/document_compressors/rankllm_rerank.py:69, in RankLLMRerank.validate_environment(cls, values)
68 elif model_enum == ModelType.GPT:
---> 69 from rank_llm.rerank.rank_gpt import SafeOpenai
70 from rank_llm.rerank.reranker import Reranker

ModuleNotFoundError: No module named 'rank_llm.rerank.rank_gpt'

During handling of the above exception, another exception occurred:

ImportError Traceback (most recent call last)
Cell In[4], line 8
3 from core.retrieval import retriever_rank_gpt
5 import rank_llm
7 retrieval_chain = query_translation(prompt_template=multi_query_prompt_template, retriever=
----> 8 retriever_rank_gpt(retriever=retriever)
9 )
11 docs = retrieval_chain.invoke({"question": "What is task decomposition for LLM agents?"})
13 pretty_print_docs(docs)

File ~/.../src/core/retrieval.py:55, in retriever_rank_gpt(retriever)
52 def retriever_rank_gpt(
53 retriever: VectorStoreRetriever,
54 ) -> VectorStoreRetriever:
...
86 "Please install it with pip install rank_llm."
87 )
89 return values

ImportError: Could not import rank_llm python package. Please install it with pip install rank_llm.

Description

The RankLLMRerank integration in LangChain, designed to rerank documents, throws an error when used with GPT (or other models) if the rank-llm package version exceeds 0.12.8. This issue arises due to breaking changes introduced in newer versions of the rank-llm package, now all of the files like rank_gpt, vicuna_reranker or zephyr_reranker were moved to the submodule rank_llm.rerank.listwise.

System Info

System Information

OS: Darwin
OS Version: Darwin Kernel Version 23.4.0: Wed Feb 21 21:45:49 PST 2024; root:xnu-10063.101.15~2/RELEASE_ARM64_T6020
Python Version: 3.10.14 (main, Mar 19 2024, 21:46:16) [Clang 15.0.0 (clang-1500.3.9.4)]

Package Information

langchain_core: 0.3.29
langchain: 0.3.14
langchain_community: 0.3.14
langsmith: 0.2.4
langchain_anthropic: 0.3.1
langchain_aws: 0.2.2
langchain_chroma: 0.2.0
langchain_experimental: 0.3.2
langchain_fireworks: 0.2.6
langchain_google_vertexai: 2.0.5
langchain_groq: 0.2.3
langchain_mistralai: 0.2.4
langchain_openai: 0.3.0
langchain_text_splitters: 0.3.5
langchain_together: 0.2.0
langchain_unstructured: 0.1.5
langchainhub: 0.1.21
langgraph_sdk: 0.1.34

Optional packages not installed

langserve

Other Dependencies

aiohttp: 3.11.11
anthropic: 0.40.0
anthropic[vertexai]: Installed. No version info available.
async-timeout: 4.0.3
boto3: 1.35.42
chromadb: 0.5.15
dataclasses-json: 0.6.7
defusedxml: 0.7.1
fastapi: 0.115.3
fireworks-ai: 0.15.7
google-cloud-aiplatform: 1.70.0
google-cloud-storage: 2.18.2
groq: 0.11.0
httpx: 0.28.1
httpx-sse: 0.4.0
jsonpatch: 1.33
langsmith-pyo3: Installed. No version info available.
numpy: 1.26.4
openai: 1.57.4
orjson: 3.10.12
packaging: 24.2
pydantic: 2.10.4
pydantic-settings: 2.7.0
PyYAML: 6.0.2
rank-llm: 0.20.3
requests: 2.32.3
requests-toolbelt: 1.0.0
SQLAlchemy: 2.0.36
tenacity: 9.0.0
tiktoken: 0.8.0
tokenizers: 0.20.1
types-requests: 2.32.0.20241016
typing-extensions: 4.12.2
unstructured-client: 0.25.9
unstructured[all-docs]: Installed. No version info available.