Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: update Firecrawl version and add FirecrawlExtractTool #4073

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

ftonato
Copy link

@ftonato ftonato commented Feb 24, 2025

Hello,

I am one of the people building Firecrawl, and it would be great to update the version to support our new API version (v1). I made a few changes, but I wasn’t able to test them.


It would be helpful if we could test it before merging to ensure everything is working as expected. Could someone help me understand how to test it?

The visual part is working as expected, but my problem is how to check the request/response from each method we have?

image

Thank you!

@HenryHengZJ
Copy link
Contributor

thanks @ftonato !

One way to test it is via Document Store:
image
https://docs.flowiseai.com/using-flowise/document-stores

optional: true,
additionalParams: true
}
// ... (other input parameters)
]
this.outputs = [
{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't change the output here, as they are meant to connect to other nodes:
image

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will revert this part!

@ftonato
Copy link
Author

ftonato commented Feb 27, 2025

Hey @HenryHengZJ,

I am still trying this approach, but every time I get the same error: Failed to preview chunks: Error: documentStoreServices.previewChunksMiddleware - Error: documentStoreServices.previewChunks - Error: documentStoreServices.splitIntoChunks - Request failed with status code 400. In the network tab, I can see that I have a 500 error.

Let me share the video I recorded so you can see what I’m doing:

CleanShot.2025-02-27.at.23.42.59.mp4

@ftonato
Copy link
Author

ftonato commented Mar 4, 2025

Hello @HenryHengZJ, did you have the chance to see my last message?

@ftonato ftonato requested a review from HenryHengZJ March 19, 2025 17:16
@ftonato ftonato changed the title [DRAFT] chore: update Firecrawl version and add FirecrawlExtractTool chore: update Firecrawl version and add FirecrawlExtractTool Mar 21, 2025
@ftonato ftonato force-pushed the refactor/update-firecrawl-api-to-v1 branch 8 times, most recently from 11d94a8 to efa02ea Compare March 21, 2025 13:32
@ftonato ftonato force-pushed the refactor/update-firecrawl-api-to-v1 branch from efa02ea to f2ab76a Compare March 21, 2025 13:35
@ftonato
Copy link
Author

ftonato commented Mar 21, 2025

Hello @HenryHengZJ & @chungyau97, I apologize for pinging you again, but I am still waiting for some clarity. Could you point me to someone who can help me with this?

@chungyau97
Copy link
Contributor

Hi @ftonato ,

I recommend that you follow this documentation and use this template as an example.

You can use this chatflow example that work on Cheerio to test it.
image

image

ftonato added 2 commits March 24, 2025 21:46
…traction

- Introduced FireCrawl-TEST.ts to validate FireCrawlLoader functionality.
- Implemented tests for basic scraping, crawling with text splitting, data extraction, and extract status retrieval.
- Enhanced error handling in FireCrawlLoader for better debugging.
@ftonato
Copy link
Author

ftonato commented Mar 24, 2025

Hello @chungyau97,

Thanks a lot for providing the schema! I was able to test it successfully and went ahead and created a test file to help with local debugging. If possible, please leave that file in place.

I’m responsible for the Firecrawl implementation and I’m watching the repo, so I’ll get notified if anyone else makes changes to those files 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants