Alexander/homepage
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
master
homepage/static/robots.txt

6 lines
65 B
Plaintext
Raw Permalink Normal View History

disallow GPTBot/Openai from crawling website
2024-07-30 14:03:32 +02:00
User-agent: GPTBot
Disallow: /
no crawling of /static/
2024-07-30 14:08:41 +02:00
User-agent: *
Disallow: /static/
Reference in New Issue Copy Permalink