Merge pull request #1675 from starknet-io/bugfix/duplicated-pages

Update robot to not index unused translations
starknet-io · Nov 20, 2023 · a70ba95 · a70ba95
2 parents bcc1dd3 + d54e5a5
commit a70ba95
Showing 1 changed file with 8 additions and 1 deletion.
diff --git a/public/robots.txt b/public/robots.txt
@@ -1,2 +1,9 @@
 User-agent: *
-Allow: /
+Allow: /
+Disallow: /ar/*
+Disallow: /de/*
+Disallow: /es/*
+Disallow: /fr/*
+Disallow: /ja/*
+Disallow: /ko/*
+Disallow: /pt/*