Configure robots.txt

wordpress時代のクローリングが残ってnew relicのログのノイズになるため、robots.txtで明示的にクローリングを禁止した
sora32127 · Apr 25, 2024 · 8f943a9 · 8f943a9
1 parent ca94d66
commit 8f943a9
Showing 1 changed file with 5 additions and 1 deletion.
diff --git a/public/robots.txt b/public/robots.txt
@@ -1,2 +1,6 @@
 User-agent: *
-Disallow: /logout
+Disallow: /logout
+Disallow: /wp-json/
+Disallow: /archives/tag/
+Disallow: /archives/*/feed
+Disallow: /wp-login.php