Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: SemaphoreDispatcher does not work with stream=True #703

Open
ederuiter opened this issue Feb 17, 2025 · 0 comments
Open

[Bug]: SemaphoreDispatcher does not work with stream=True #703

ederuiter opened this issue Feb 17, 2025 · 0 comments
Labels
🐞 Bug Something isn't working 🩺 Needs Triage Needs attention of maintainers

Comments

@ederuiter
Copy link

crawl4ai version

2025-feb-alpha-1

Expected Behavior

No error

Current Behavior

The SemaphoreDispatcher does not implement the run_urls_stream method, but this is required as it is called from

async for task_result in dispatcher.run_urls_stream(crawler=self, urls=urls, config=config):
in case the config option stream is set to True
So this will error out.

Is this reproducible?

Yes

Inputs Causing the Bug

Any

Steps to Reproduce

Code snippets

OS

any

Python version

any

Browser

No response

Browser version

No response

Error logs & Screenshots (if applicable)

No response

@ederuiter ederuiter added 🐞 Bug Something isn't working 🩺 Needs Triage Needs attention of maintainers labels Feb 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐞 Bug Something isn't working 🩺 Needs Triage Needs attention of maintainers
Projects
None yet
Development

No branches or pull requests

1 participant