page endpoint

[sushidelivery]

Hello! Thank you for your amazing contribution!

I would wanted to ask about using proxy with email & password?
How can I configure that?

Many thanks!

---

i got it. Just enter like this "http://user:[email protected]:3128/"
Please, reply on the second question;)

[sushidelivery]

and second question, is there anyway I can add a checkpoint, so if i scraped for first time, and then when i scrape for the second time it will start from page where first time ended?

[tasos-py]

No, there is no starting page option. You could implement this feature, and I would help, but it won't be easy. You would have to modify the ._next_page() method of every class that inherits from SearchEngine separately, because you would have to be able to ask for a specific pagination element and each search engine has different HTML structure.