Add option to set request parameters for MathpixPDFLoader

[juliuskittler]

Feature request

Status Quo

Currently, the MathpixPDFLoader sets its data property as follows:

    @property
    def data(self) -> dict:
        options = {"conversion_formats": {self.processed_file_format: True}}
        return {"options_json": json.dumps(options)}

However, there are a variety of additional useful parameters that can be defined in the MathPix API calls: https://docs.mathpix.com/#request-parameters-6 For instance, this includes enable_tables_fallback, numbers_default_to_math, enable_spell_check etc.

It would be great if these parameters could be specified as keyword arguments directly in the MathpixPDFLoader.

Motivation

For instance, tables do not always get parsed properly in PDFs and therefore in some cases it can make sense to set enable_tables_fallback=True.

Your contribution

TBD

[dosubot]

🤖

Hi @juliuskittler, good to see you again!

Thank you for your feature request. I agree that adding the ability to set request parameters for the MathpixPDFLoader would be a useful enhancement to the LangChain framework.

Based on your suggestion, we can modify the send_pdf method in the MathpixPDFLoader class to accept a dictionary of parameters and include these in the POST request to the Mathpix API. Here's a possible implementation:

def send_pdf(self, params: Optional[Dict[str, Any]] = None) -> str:
    with open(self.file_path, "rb") as f:
        files = {"file": f}
        response = requests.post(
            self.url, headers=self._mathpix_headers, files=files, data=self.data, params=params
        )
    response_data = response.json()
    if "pdf_id" in response_data:
        pdf_id = response_data["pdf_id"]
        return pdf_id
    else:
        raise ValueError("Unable to send PDF to Mathpix.")

In this code, params is a dictionary that contains the request parameters you want to set. These parameters are then included in the POST request to the Mathpix API by passing them to the requests.post method.

You can then use this method like this:

params = {
    'enable_tables_fallback': True,
    'numbers_default_to_math': True,
    'enable_spell_check': True
}
pdf_id = loader.send_pdf(params)

This will send a POST request to the Mathpix API with the specified parameters.

Please note that this is just a suggestion and the actual implementation may vary based on the LangChain codebase and the specific requirements of the project.

I hope this helps! If you have any other questions or need further clarification, feel free to ask.

Best,
Dosu

Sources

• libs/langchain/langchain/document_loaders/pdf.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[dosubot]

Hi, @juliuskittler,

I'm helping the LangChain team manage their backlog and am marking this issue as stale. The issue you opened requested the addition of an option to set request parameters for the MathpixPDFLoader, allowing users to customize API calls for improving the parsing of PDFs. In response, a detailed suggestion was provided on how to implement this feature, including modifying the send_pdf method in the MathpixPDFLoader class to accept a dictionary of parameters and include them in the POST request to the Mathpix API. The suggestion also included example usage of the proposed method.

Could you please confirm if this issue is still relevant to the latest version of the LangChain repository? If it is, please let the LangChain team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or the issue will be automatically closed in 7 days.

Thank you!

[baskaryan]

Resolved in #13950