example robots.txt

creDocker · Feb 10, 2025 · 971bcb1 · 971bcb1
1 parent c59e5a4
commit 971bcb1
Showing 1 changed file with 8 additions and 0 deletions.
diff --git a/cre/robots.txt b/cre/robots.txt
@@ -0,0 +1,8 @@
+User-agent: *
+Disallow: /
+
+Crawl-delay: 60
+User-agent: Bingbot
+User-Agent: Googlebot
+Disallow:
+Sitemap: http://www.tambora.org/sitemap.xml