-
|
I'm trying to stop the crawler after a given condition is met. I am following this documentation but it's not working as intended. The crawling requests do stop, but the process doesn't terminate and instead it hangs there for a minute or two, and then the program crashes. In the logs I get this:
When I go into the code for the playwright crawler and the basic crawler I am unable to find the stop method that the documentation refers to, I've also checked the BeautifulSoup crawler which is what is used in the docs. I am not sure if I'm overlooking something or if the documentation is maybe outdated? If outdated, what is the correct way of going about this? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
Hey @jkumz Try testing this on version 0.5.0b30. This is functionality that will be added in version 0.5.0, it's just that the documentation is already updated, but the release hasn't happened yet. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks, that worked |
Beta Was this translation helpful? Give feedback.
Hey @jkumz
Try testing this on version 0.5.0b30.
This is functionality that will be added in version 0.5.0, it's just that the documentation is already updated, but the release hasn't happened yet.