core: fix ChatPromptTemplate doesn't accept PDF data as bytes

[rkwan05]

Description:

• Functionality (chat.py): added if statement in ChatPromptTemplate's from_template() that catches data in mime_type: {pdf_data} format
  • extracts text from pdf and inserts it into the prompt
• Unit Test (test_chat.py): added test that passes in pdf data to ChatPromptTemplate

Issue: #27346
Dependencies:

• base64: Decodes pdf data in chat.py > extract_pdf_text() and encodes pdf data in text_chat.py > test_create_pdf_chat_prompt()
• io: Creates an in-memory binary stream to handle pdf data in chat.py > extract_pdf_text()
• pypdf: To import PDFReader to read the pdf file in chat.py > extract_pdf_text()

Lint and test:

Ran make format, make lint, and make test from the root of libs\core. All checks passed successfully.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
langchain	❌ Failed (Inspect)			Nov 26, 2024 2:33am

[efriis]

which chat model supports this?

[rkwan05]

Hi Efriis, I just wanted to make sure that we understood the question correctly! If you’re asking which chat models this implementation would be compatible with, it should work with all chat models because it is sending the pdf data as text. But please let me know if you were looking for something else!

[efriis]

closing because this isn't desirable in the prompt template. in general, the right process here would be parse the pdfs using something like this, and plumb that text into your prompt

this implementation would import PyPDF parsing for anyone trying to use a prompt template, which isn't mergable