Skip to content

Commit

Permalink
Add a note on the AI article about non AI scrapers
Browse files Browse the repository at this point in the history
  • Loading branch information
themkat committed Dec 1, 2024
1 parent fb4cdd9 commit 599e507
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion org/_posts/2024-08-16-ai_block_crawlers.org
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,10 @@ tags: ai opinions

Have you ever wondered about how you can block "artificial intelligence" (AI) scrapers/crawlers/bots from stealing your web sites' content? So have I. Like many others, I have been annoyed at the misuse of these technological solutions. Ranging from big companies using our content without permission, to idiots on social media using generated ChatGPT text to sound profound. If you are curious on deranged ranting on AI, or more importantly how you can tell them to f**k off, then read on!


In this article, I will refer to these scrapers, bots etc. as simply crawlers. This is the term that makes most sense to me.

*NOTE!!! Updated December 1st 2024. This article is SPECIFICALLY ABOUT AI RELATED BOTS/CRAWLERS! Other crawlers like Planet RSS sites, scrapers used for sharing links on news sites etc. is almost always a good thing in comparison. I have nothing against you polling the RSS feed or doing anything related to automatically share links to my content. Copying entire articles to other places is where we have a problem, as well as you making a commercial closed source AI with my content that only serves you...*


* The short answer
You (probably) can't. There are some ways, but nothing in life is guaranteed (except death and taxes).
Expand Down

0 comments on commit 599e507

Please sign in to comment.