Consider https://arstechnica.com/robots.txt or https://www.nytimes.com/robots.txt and how they block all the stupid AI models from being able to scrape for free.
Consider https://arstechnica.com/robots.txt or https://www.nytimes.com/robots.txt and how they block all the stupid AI models from being able to scrape for free.
Perhaps this will help your understanding of my first point.
https://gizmodo.com/former-openai-board-member-sam-altman-chatgpt-1851506252