It's not really something we can do, sadly. Reddit closing it's API was more about getting money than actually stopping it's use as a training set.
Having an allow-list is a start though, as it means that a company can't just make an instance and suck all the data out through that. Common corporate crawlers could be added to the robots.txt, but that would mean that you might not be able to find lemmy instances in search results. We could make it against ToS, but what are we going to do, sue the massive corporation? They have plenty of lawyer and payout money, so very little would fundamentally change.
Ultimately, if content can be served to us, it can be served to them.
It's not really something we can do, sadly. Reddit closing it's API was more about getting money than actually stopping it's use as a training set.
Having an allow-list is a start though, as it means that a company can't just make an instance and suck all the data out through that. Common corporate crawlers could be added to the robots.txt, but that would mean that you might not be able to find lemmy instances in search results. We could make it against ToS, but what are we going to do, sue the massive corporation? They have plenty of lawyer and payout money, so very little would fundamentally change.
Ultimately, if content can be served to us, it can be served to them.