Draft: community: add pull_md tool for converting URLs to Markdown

[chigwell]

• PR title: community: add pull_md tool for converting URLs to Markdown

• PR message:

  • Description: This pull request adds the pull_md tool to the Langchain community package, enabling the conversion of URLs to Markdown format. It's particularly effective for processing web pages built with dynamic JavaScript frameworks like React, Angular, and Vue.js. By leveraging the pull.md service, pull_md retrieves fully rendered Markdown without using local server resources, simplifying the conversion of complex web pages into Markdown for documentation and educational projects. The tool enhances Langchain's functionality.
  • Dependencies: Requires the pull_md package from PyPI.
  • Twitter handle: https://x.com/eugeneevstafev?s=21

• Add tests and docs:

  1. Added unit tests that do not rely on network access.
  2. Included an example notebook demonstrating its use, located in the docs/docs/integrations directory.

• Lint and test: Ran make format, make lint, and make test from the root of the package(s) I've modified. All checks pass as per the contribution guidelines.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
langchain	❌ Failed (Inspect)			Jan 3, 2025 5:21pm

[efriis]

Hey! This adds a net-new community integration or feature, which has been replaced by dedicated integration packages. I'll close this PR, and would recommend reopening with just docs updates, as well as registering your package in libs/packages.yml! We'll be able to review simple PRs that only modify these two things much faster :)

Here's the guide, and if you have questions, feel free to leave them in the comments on those pages so others can see them! https://python.langchain.com/docs/contributing/how_to/integrations/