Welcome to animdl Discussions!

[justfoolingaround]

Sup' sup' scraper devs and users!

We're now using GitHub discussions to discuss matters related but not limited to this project. You can:

• Ask any programming related questions. (it could be anything related but you might not get a good answer here of all places!)
• Share your ideas!
• Engage with other community members.

The only requirement is to be civil.

Note: You can engage by going and sharing your endeavors in the comments. (No judgements, I promise!)

Let's talk about scraping in general. Web scraping is quite effective and flexible method of collecting data from the internet. Once you've mastered this art, you won't ever have to search for "API" in a site whose data you want to use! While this project focuses on anime streaming site scraping, it does not mean that we do not know how to scrape any other sites. Anime streaming sites may even be a challenging but efficient way to master scraping in general because the protections used by these sites are unique, not standard and tricky to deal with.

If you do not like anime or streaming sites or both, you can even scrape movies from torrents and stream it via mediums like peerflix.

Point is, it is an absolutely interesting programming expertise which goes hand in hand with automation. It is not late to learn any of this! You could be an old man in your deathbed and still scrape those juicy URLs from your favorite sites and ease your pains by pressing a single keystroke!

If you go crazy with web scraping, you might even master the more powerful stuff!

• Websockets,
• Asynchronous programming,
• Working of a HTTP Protocol,
• DNS and Proxies,
• Cryptography (AES, DES and other generally used ciphers.)
• Child process handling,
• Malware creation (Cookie, authentication extraction from user computers),
  And finally,
• Mastery of whatever programming language you're scraping in!

Of course, you need to constantly challenge yourself and go deep into the rabbit hole of these simple or complex topics. But that's the fun.

It could be your turn to go from a script kiddie to someone who calls others script kiddies:tm:!

[port19x]

I cannot understate the importance of regular expressions in webscraping.
Never use an html parser and traverse some DOM tree when you can match a simple pattern.

One of the ways to get into more advanced webscraping is to learn web hacking with a tool like burpsuite.
Cryptography is a bitch

[gamechanger2030]

i am new to python scraping, saw your code today, amazing work, will try to learn from it