Should we block OpenAI from scrapping the server?
Should we block OpenAI from scrapping the server?
platform.openai.com /docs/gptbot
Right now, robots.txt on lemmy.ca is configured this way
User-Agent: *
Disallow: /login
Disallow: /login_reset
Disallow: /settings
Disallow: /create_community
Disallow: /create_post
Disallow: /create_private_message
Disallow: /inbox
Disallow: /setup
Disallow: /admin
Disallow: /password_change
Disallow: /search/
Disallow: /modlog
Would it be a good idea privacy-wise to deny GPTBot from scrapping content from the server?
User-agent: GPTBot
Disallow: /
Thanks!
4 crossposts
You're viewing a single thread.
All Comments
18 comments
Are they even respecting those files?
But yeah, sure, it's worth trying!
5 0 ReplyIt's from the official documentation.
3 0 ReplyWorth trying for what reason?
1 0 Reply
18 comments
Scroll to top