Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

community: add citation support to Perplexity chat model #28004

Closed
wants to merge 3 commits into from
Closed

community: add citation support to Perplexity chat model #28004

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
aa9437f
feat(chat_models): add citation support to Perplexity chat model
Nov 9, 2024
76cbca7
feat(chat_models): add citation support to Perplexity chat model
Nov 9, 2024
2dd84db
updated test notebook for perplexity chat
Nov 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 22 additions & 20 deletions docs/docs/integrations/chat/perplexity.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": 4,
"id": "70cf04e8-423a-4ff6-8b09-f11fb711c817",
"metadata": {
"ExecuteTime": {
Expand All @@ -103,7 +103,7 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": 6,
"id": "8199ef8f-eb8b-4253-9ea0-6c24a013ca4c",
"metadata": {
"ExecuteTime": {
Expand All @@ -116,10 +116,10 @@
{
"data": {
"text/plain": [
"'The Higgs Boson is an elementary subatomic particle that plays a crucial role in the Standard Model of particle physics, which accounts for three of the four fundamental forces governing the behavior of our universe: the strong and weak nuclear forces, electromagnetism, and gravity. The Higgs Boson is important for several reasons:\\n\\n1. **Final Elementary Particle**: The Higgs Boson is the last elementary particle waiting to be discovered under the Standard Model. Its detection helps complete the Standard Model and further our understanding of the fundamental forces in the universe.\\n\\n2. **Mass Generation**: The Higgs Boson is responsible for giving mass to other particles, a process that occurs through its interaction with the Higgs field. This mass generation is essential for the formation of atoms, molecules, and the visible matter we observe in the universe.\\n\\n3. **Implications for New Physics**: While the detection of the Higgs Boson has confirmed many aspects of the Standard Model, it also opens up new possibilities for discoveries beyond the Standard Model. Further research on the Higgs Boson could reveal insights into the nature of dark matter, supersymmetry, and other exotic phenomena.\\n\\n4. **Advancements in Technology**: The search for the Higgs Boson has led to significant advancements in technology, such as the development of artificial intelligence and machine learning algorithms used in particle accelerators like the Large Hadron Collider (LHC). These advancements have not only contributed to the discovery of the Higgs Boson but also have potential applications in various other fields.\\n\\nIn summary, the Higgs Boson is important because it completes the Standard Model, plays a crucial role in mass generation, hints at new physics phenomena beyond the Standard Model, and drives advancements in technology.\\n'"
"'There are no specific scientific studies mentioned in the provided sources that directly link soy isoflavone supplements to a positive effect on telomere length in humans. However, some studies suggest that soybean extract can increase telomerase reverse transcriptase protein expression in pancreatic β-cells of diabetes mellitus-induced rats, which might indirectly support the idea that soy components could influence telomere dynamics positively[4]. For a definitive answer, further research specifically focusing on soy isoflavones and their impact on human telomeres would be necessary. \\n\\nIn general, the literature does suggest that certain dietary components and supplements can positively affect telomere length, such as vitamins C and D, polyunsaturated fatty acids, and specific dietary patterns like the Mediterranean diet[1][2][5]. However, the direct effect of soy isoflavones on human telomeres remains unexplored in the sources provided.\\n\\nCitations:\\n[1] https://www.mdpi.com/2072-6643/16/17/2835\\n[2] https://onlinelibrary.wiley.com/doi/full/10.1002/fsn3.3851\\n[3] https://www.apco.co.th/storage/newsroom/research-and-development/2024/dietary-supplement/food-science.pdf\\n[4] https://mji.ui.ac.id/journal/index.php/mji/article/view/1732/1582\\n[5] https://www.mdpi.com/2072-6643/16/15/2525\\n'"
]
},
"execution_count": 4,
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
Expand All @@ -130,7 +130,7 @@
"prompt = ChatPromptTemplate.from_messages([(\"system\", system), (\"human\", human)])\n",
"\n",
"chain = prompt | chat\n",
"response = chain.invoke({\"input\": \"Why is the Higgs Boson important?\"})\n",
"response = chain.invoke({\"input\": \"Are there any scientific studies backing that taking soy isoflavone supplements could have a positive effect on telomere length in humans?\"})\n",
"response.content"
]
},
Expand Down Expand Up @@ -183,7 +183,7 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": 5,
"id": "025be980-e50d-4a68-93dc-c9c7b500ce34",
"metadata": {
"ExecuteTime": {
Expand All @@ -197,27 +197,29 @@
"name": "stdout",
"output_type": "stream",
"text": [
"Here is a list of some famous tourist attractions in Pakistan:\n",
"There are no specific scientific studies directly linking the consumption of soy isoflavone supplements to a positive effect on telomere length in humans. However, there are studies that suggest certain dietary components and supplements can influence telomere length positively, such as:\n",
"\n",
"1. **Mylife/Mylife100® Dietary Supplement**: This study found that a dietary supplement derived from five edible plants, including soy protein, significantly increased mean telomere length in Thai adults over an 8-week period[4][5].\n",
"\n",
"2. **Telomere Length Dynamics**: Various studies have shown that certain nutrients like vitamins C, D, and E, as well as polyunsaturated fatty acids, can help maintain or even elongate telomeres by reducing oxidative stress and inflammation[2][3].\n",
"\n",
"3. **Soybean Extract**: While not specifically focused on soy isoflavones, a study on soybean extract found that it increased telomerase reverse transcriptase protein expression in pancreatic β-cells of diabetes mellitus-induced rats, which could indirectly suggest a potential positive effect on telomeres in certain contexts[1].\n",
"\n",
"1. **Minar-e-Pakistan**: A 62-meter high minaret in Lahore that represents the history of Pakistan.\n",
"2. **Badshahi Mosque**: A historic mosque in Lahore with a capacity of 10,000 worshippers.\n",
"3. **Shalimar Gardens**: A beautiful garden in Lahore with landscaped grounds and a series of cascading pools.\n",
"4. **Pakistan Monument**: A national monument in Islamabad representing the four provinces and three districts of Pakistan.\n",
"5. **National Museum of Pakistan**: A museum in Karachi showcasing the country's cultural history.\n",
"6. **Faisal Mosque**: A large mosque in Islamabad that can accommodate up to 300,000 worshippers.\n",
"7. **Clifton Beach**: A popular beach in Karachi offering water activities and recreational facilities.\n",
"8. **Kartarpur Corridor**: A visa-free border crossing and religious corridor connecting Gurdwara Darbar Sahib in Pakistan to Gurudwara Sri Kartarpur Sahib in India.\n",
"9. **Mohenjo-daro**: An ancient Indus Valley civilization site in Sindh, Pakistan, dating back to around 2500 BCE.\n",
"10. **Hunza Valley**: A picturesque valley in Gilgit-Baltistan known for its stunning mountain scenery and unique culture.\n",
"However, it is crucial to note that these studies do not directly address the specific impact of soy isoflavone supplements on human telomere length. Further research would be needed to definitively establish any such relationship\n",
"\n",
"These attractions showcase the rich history, diverse culture, and natural beauty of Pakistan, making them popular destinations for both local and international tourists.\n"
"Citations:\n",
"[1] https://mji.ui.ac.id/journal/index.php/mji/article/view/1732/1582\n",
"[2] https://www.mdpi.com/2072-6643/16/17/2835\n",
"[3] https://www.mdpi.com/2072-6643/16/15/2525\n",
"[4] https://onlinelibrary.wiley.com/doi/full/10.1002/fsn3.3851\n",
"[5] https://www.apco.co.th/storage/newsroom/research-and-development/2024/dietary-supplement/food-science.pdf"
]
}
],
"source": [
"chat = ChatPerplexity(temperature=0.7, model=\"llama-3.1-sonar-small-128k-online\")\n",
"prompt = ChatPromptTemplate.from_messages(\n",
" [(\"human\", \"Give me a list of famous tourist attractions in Pakistan\")]\n",
" [(\"human\", \"Are there any scientific studies backing that taking soy isoflavone supplements could have a positive effect on telomere length in humans?\")]\n",
")\n",
"chain = prompt | chat\n",
"for chunk in chain.stream({}):\n",
Expand All @@ -241,7 +243,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.12"
"version": "3.12.6"
}
},
"nbformat": 4,
Expand Down
90 changes: 61 additions & 29 deletions libs/community/langchain_community/chat_models/perplexity.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,22 +46,9 @@


class ChatPerplexity(BaseChatModel):
"""`Perplexity AI` Chat models API.

To use, you should have the ``openai`` python package installed, and the
environment variable ``PPLX_API_KEY`` set to your API key.
Any parameters that are valid to be passed to the openai.create call can be passed
in, even if not explicitly saved on this class.

Example:
.. code-block:: python

from langchain_community.chat_models import ChatPerplexity

chat = ChatPerplexity(
model="llama-3.1-sonar-small-128k-online",
temperature=0.7,
)
"""Modified `Perplexity AI` Chat models API with citation support.

This version includes citations in the output when available from the API response.
"""

client: Any = None #: :meta private:
Expand All @@ -74,18 +61,12 @@ class ChatPerplexity(BaseChatModel):
pplx_api_key: Optional[str] = Field(
default_factory=from_env("PPLX_API_KEY", default=None), alias="api_key"
)
"""Base URL path for API requests,
leave blank if not using a proxy or service emulator."""
request_timeout: Optional[Union[float, Tuple[float, float]]] = Field(
None, alias="timeout"
)
"""Timeout for requests to PerplexityChat completion API. Default is None."""
max_retries: int = 6
"""Maximum number of retries to make when generating."""
streaming: bool = False
"""Whether to stream the results or not."""
max_tokens: Optional[int] = None
"""Maximum number of tokens to generate."""

model_config = ConfigDict(
populate_by_name=True,
Expand Down Expand Up @@ -204,9 +185,9 @@ def _convert_delta_to_message_chunk(
elif role == "tool" or default_class == ToolMessageChunk:
return ToolMessageChunk(content=content, tool_call_id=_dict["tool_call_id"])
elif role or default_class == ChatMessageChunk:
return ChatMessageChunk(content=content, role=role) # type: ignore[arg-type]
return ChatMessageChunk(content=content, role=role)
else:
return default_class(content=content) # type: ignore[call-arg]
return default_class(content=content)

def _stream(
self,
Expand All @@ -224,15 +205,45 @@ def _stream(
stream_resp = self.client.chat.completions.create(
model=params["model"], messages=message_dicts, stream=True
)

# Track whether we've seen the last chunk
final_chunk_seen = False
citations = []

for chunk in stream_resp:
if not isinstance(chunk, dict):
chunk = chunk.dict()
if len(chunk["choices"]) == 0:
chunk_dict = chunk.dict()
else:
chunk_dict = chunk

# Store citations if they exist
if "citations" in chunk_dict:
citations = chunk_dict["citations"]

if len(chunk_dict["choices"]) == 0:
continue
choice = chunk["choices"][0]

choice = chunk_dict["choices"][0]

# Check if this is the final chunk
if choice.get("finish_reason") is not None:
final_chunk_seen = True

# Create message chunk
chunk = self._convert_delta_to_message_chunk(
choice["delta"], default_chunk_class
)

# Add citations if this is the final chunk and we have citations
if final_chunk_seen and citations:
citation_text = "\n\nCitations:\n" + "\n".join(
f"[{i+1}] {citation}" for i, citation in enumerate(citations)
)
chunk = self._convert_delta_to_message_chunk(
{"role": "assistant", "content": citation_text},
default_chunk_class
)

finish_reason = choice.get("finish_reason")
generation_info = (
dict(finish_reason=finish_reason) if finish_reason is not None else None
Expand All @@ -243,6 +254,14 @@ def _stream(
run_manager.on_llm_new_token(chunk.text, chunk=chunk)
yield chunk

def _append_citations(self, content: str, citations: List[str]) -> str:
"""Append citations to the message content."""
if citations:
content += "\n\nCitations:\n"
for i, citation in enumerate(citations, 1):
content += f"[{i}] {citation}\n"
return content

def _generate(
self,
messages: List[BaseMessage],
Expand All @@ -256,12 +275,25 @@ def _generate(
)
if stream_iter:
return generate_from_stream(stream_iter)

message_dicts, params = self._create_message_dicts(messages, stop)
params = {**params, **kwargs}
response = self.client.chat.completions.create(
model=params["model"], messages=message_dicts
)
message = AIMessage(content=response.choices[0].message.content)

# Get the response content
content = response.choices[0].message.content

# Get citations if available (assuming response is converted to dict)
response_dict = response.dict() if hasattr(response, 'dict') else response
citations = response_dict.get('citations', [])

# Append citations to content if available
if citations:
content = self._append_citations(content, citations)

message = AIMessage(content=content)
return ChatResult(generations=[ChatGeneration(message=message)])

@property
Expand All @@ -277,4 +309,4 @@ def _invocation_params(self) -> Mapping[str, Any]:
@property
def _llm_type(self) -> str:
"""Return type of chat model."""
return "perplexitychat"
return "perplexitychat"
Loading