-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathrobots.txt
110 lines (108 loc) · 1.56 KB
/
robots.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
# This is necessary to prevent the raw pages being used for spam
User-agent: *
Disallow: dump.php
Disallow: del.php
Disallow: raw.php
Disallow: ref.php
Disallow: config.php
Disallow: functions.php
Disallow: /00
Disallow: /01
Disallow: /02
Disallow: /03
Disallow: /04
Disallow: /05
Disallow: /06
Disallow: /07
Disallow: /08
Disallow: /09
Disallow: /10
Disallow: /11
Disallow: /12
Disallow: /13
Disallow: /14
Disallow: /15
Disallow: /16
Disallow: /17
Disallow: /18
Disallow: /19
Disallow: /20
Disallow: /21
Disallow: /22
Disallow: /23
Disallow: /24
Disallow: /25
Disallow: /26
Disallow: /27
Disallow: /28
Disallow: /29
Disallow: /30
Disallow: /31
Disallow: /32
Disallow: /33
Disallow: /34
Disallow: /35
Disallow: /36
Disallow: /37
Disallow: /38
Disallow: /39
Disallow: /40
Disallow: /41
Disallow: /42
Disallow: /43
Disallow: /44
Disallow: /45
Disallow: /46
Disallow: /47
Disallow: /48
Disallow: /49
Disallow: /50
Disallow: /51
Disallow: /52
Disallow: /53
Disallow: /54
Disallow: /55
Disallow: /56
Disallow: /57
Disallow: /58
Disallow: /59
Disallow: /60
Disallow: /61
Disallow: /62
Disallow: /63
Disallow: /64
Disallow: /65
Disallow: /66
Disallow: /67
Disallow: /68
Disallow: /69
Disallow: /70
Disallow: /71
Disallow: /72
Disallow: /73
Disallow: /74
Disallow: /75
Disallow: /76
Disallow: /77
Disallow: /78
Disallow: /79
Disallow: /80
Disallow: /81
Disallow: /82
Disallow: /83
Disallow: /84
Disallow: /85
Disallow: /86
Disallow: /87
Disallow: /88
Disallow: /89
Disallow: /90
Disallow: /91
Disallow: /92
Disallow: /93
Disallow: /94
Disallow: /95
Disallow: /96
Disallow: /97
Disallow: /98
Disallow: /99