Repositories list

• scrapy-pipelines

  Public
  A collection of pipelines for Scrapy

  pipelinespython3scrapy

  Python

  •

  GNU General Public License v3.0

  •4•16•25•10•Updated Feb 22, 2025Feb 22, 2025

• scrapy-cookies

  Public
  A middleware of cookies persistence for Scrapy

  pythonmiddlewarecookies+ 1scrapy

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •15•27•44•3•Updated Feb 22, 2025Feb 22, 2025

• r18

  Public
  A scrapy spider for R18

  python3scrapyr18

  Python

  •

  GNU General Public License v3.0

  •6•15•34•12•Updated Aug 6, 2024Aug 6, 2024

• scrapy-httpproxy

  Public
  A middleware enhanced the default httpproxy one for Scrapy

  pythonmiddlewareproxy+ 1scrapy

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •3•4•0•1•Updated Dec 15, 2023Dec 15, 2023

• docker-privoxy

  Public
  Dockerfile

  •

  GNU General Public License v3.0

  •0•1•1•2•Updated Oct 19, 2023Oct 19, 2023

• shadowsocks

  Public
  Python

  •

  Apache License 2.0

  •19k•1•0•1•Updated Apr 4, 2023Apr 4, 2023

• pyppeteer

  Public
  Headless chrome/chromium automation library (unofficial port of puppeteer)

  Python

  •

  Other

  •371•1•0•1•Updated Apr 4, 2023Apr 4, 2023

• scrapy-httpcache

  Public
  A downloader middleware to cache http response into multiple backends

  Python

  •1•2•2•0•Updated Oct 18, 2021Oct 18, 2021

• scrapedia.github.io

  Public
  GNU General Public License v3.0

  •0•1•0•0•Updated Aug 22, 2020Aug 22, 2020

• scrapy-useragents

  Public
  A downloader middleware to change user-agent of scrapy

  pythonmiddlewarescrapy+ 1useragent

  Python

  •5•20•2•0•Updated Jun 1, 2019Jun 1, 2019