The Fediverse is a great system for preventing bad actors from disrupting "real" human-human conversations, because all of the mods, developers and admins are all working out of a desire to connect people (as opposed to "trust and safety" teams more concerned about user retention).
Right now it seems that the Fediverses main protection is that it just isn't a juicy enough target for wide scale spam and bad faith agenda pushers.
But assuming the Fediverse does grow to a significant scale, what (current or future) mechanisms are/could be in place to fend off a flood of AI slop that is hard to distinguish from human? Even the most committed instance admins can only do so much.
For example, I have a feeling all "good" instances in the near future will eventually have to turn on registration applications and only federate with other instances that do the same. But it's not crazy to imagine that GPT could soon outmaneuver most registration questions which means registrations will only slow the growth of the problem but not manage it long-term.
Any thoughts on this topic?
There are two groups here, bots, and bad actors. We've found that these measures have mostly stopped them both.
Bots
- Registration applications. Its been extremely easy to differentiate bots from real people by asking a series of simple questions, and only let the real people in.
- Reports: so that mods / admins can see them quickly.
- Blocking open-signup servers that don't have required applications, that usually serve as spam-attacks against the whole fediverse.
Some bots still get through occasionally, but not many compared to before. And some servers have more "lax" application questions, so they let more through.
Bad actors
- Registration applications. Most of the trolls are of a temperament where they refuse to do the work of answering questions earnestly. They can't help themselves but give obviously trolling answers, if they do even bother to do that work at all.
- Reports: same as above.
- Ban + remove. Mods and admins can ban and remove all a person's content at the click of a button. So even if the troll did the work of getting past the front door, then all their work is nullified by an action that takes less than 5 seconds. So they wasted much more of their time, than they did for admins, and accomplished nothing lasting.
Great response, thank you. My concern is more so focused on future measures; what happens if/when registration applications are answerable by a bot? It's not hard to imagine. What happens when a GPT powered bot leaves totally "normal" unique comments 90% of the time, but occasionally recommends a product or pushes a political agenda?
All I can say is that in practice, bots can't answer most simple questions in a believable way, especially questions that require actual personal opinions, or that require any context outside of what they were asked.
The most we've seen is that people created seemingly lemmy-specific signup bots, but they always answer questions in the same transparent way.
The blogspam bots that have gotten through (not for many months now here on lemmy.ml) are all transparent, because they all post links to the same domain. All it takes is one report, and we can remove their entire history.
No one can mark an account as a bot one except you, so maybe its an app that keeps setting it.
i only use firefox; are the instance admins able to set it on your account? (because that would make sense)
when i first joined lemmy, i didn't understand how it worked so i would sign up with one instance and; when i could no longer up/downvote; i switched to another instances. it eventually led me to joining .ml and it was here i learned about the bot account setting and saw that it was set on the old accounts that i don't use anymore and i've always wondered why.
"The fediverse" really can't. That's just the reality of a decentralized system. It's going to be up to individual instances to sort it out.
But that's a good thing, because what it means is that different instances can and will try different approaches, and between them, they'll sooner or later hit on the one(s) that will be most effective.
Ban it outright in the rules of individual instances, bully AI piglets for printing the lowest-value content online in the same way NFT goobers are ostracised, run AI image and writing detectors on suspect posts. The common denominator of any AI post is that it's going to be shit and it should just be treated like someone repeatedly posting a Lorem ipsum copypasta or spam email.
I don't have the foggiest idea.
And really, if I did have a good idea, I wouldn't post it publicly anyway. That'd just be tipping my hand to the astroturfers.
The fediverse architecture was built from the beginning to allow instance-by-instance exercise of discretion to mute any systemic effects that could take over the network as a whole.
This was I think oriented toward limiting swarming behavior from trolls, but I think it also applies to AI bots.
Right now it seems that the Fediverses main protection is that it just isn’t a juicy enough target for wide scale spam and bad faith agenda pushers.
If you ask me they are already here right now, but I think it's not the architecture of the fediverse, but the judgment of individual mods that have let us down in this case.
