Hey folks!
I made a short post last night explaining why image uploads had been disabled. This was in the middle of the night for me, so I did not have time to go into a lot of detail, but I'm writing a more detailed post now to clear up where we are now and where we plan to go.
What's the problem?
As shared by the lemmy.world team, over the past few days, some people have been spamming one of their communities with CSAM images. Lemmy has been attacked in various ways before, but this is clearly on a whole new level of depravity, as it's first and foremost an attack on actual victims of child abuse, in addition to being an attack on the users and admins on Lemmy.
What's the solution?
I am putting together a plan, both for the short term and for the longer term, to combat and prevent such content from ever reaching lemm.ee servers.
For the immediate future, I am taking the following steps:
1) Image uploads are completely disabled for all users
This is a drastic measure, and I am aware that it's the opposite of what many of our users have been hoping, but at the moment, we simply don't have the necessary tools to safely handle uploaded images.
2) All images which have federated in from other instances will be deleted from our servers, without any exception
At this point, we have millions of such images, and I am planning to just indiscriminately purge all of them. Posts from other instances will not be broken after the deletion, the deleted images will simply be loaded directly from other instances.
3) I will apply a small patch to the Lemmy backend running on lemm.ee to prevent images from other instances from being downloaded to our servers
Lemmy has always loaded some images directly from other servers, while saving other images locally to serve directly. I am eliminating the second option for the time being, forcing all images uploaded on external instances to always be loaded from those servers. This will somewhat increase the amount of servers which users will fetch images from when opening lemm.ee, which certainly has downsides, but I believe this is preferable to opening up our servers to potentially illegal content.
For the longer term, I have some further ideas:
4) Invite-based registrations
I believe that one of the best ways to effectively combat spam and malicious users is to implement an invite system on Lemmy. I have wanted to work on such a system ever since I first set up this instance, but real life and other things have been getting in the way, so I haven't had a chance. However, with the current situation, I believe this feature is more important then ever, and I'm very hopeful I will be able to make time to work on it very soon.
My idea would be to grant our users a few invites, which would replenish every month if used. An invite will be required to sign up on lemm.ee after that point. The system will keep track of the invite hierarchy, and in extreme cases (such as spambot sign-ups), inviters may be held responsible for rule breaking users they have invited.
While this will certainly create a barrier of entry to signing up on lemm.ee, we are already one of the biggest instances, and I think at this point, such a barrier will do more good than harm.
5) Account requirements for specific activities
This is something that many admins and mods have been discussing for a while now, and I believe it would be an important feature for lemm.ee as well. Essentially, I would like to limit certain activities to users which meet specific requirements (maybe account age, amount of comments, etc). These activities might include things like image uploads, community creation, perhaps even private messages.
This could in theory limit creation of new accounts just to break rules (or laws).
6) Automated ML based NSFW scanning for all uploaded images
I think it makes sense to apply automatic scanning on all images before we save them on our servers, and if it's flagged as NSFW, then we don't accept the upload. While machine learning is not 100% accurate and will produce false positives, I believe this is a trade-off that we simply need to accept at this point. Not only will this help against any potential CSAM, it will also help us better enforce our "no pornography" rule.
This would potentially also allow us to resume caching images from other instances, which will improve both performance and privacy on lemm.ee.
With all of the above in place, I believe we will be able to re-enable image uploads with a much higher degree of safety. Of course, most of these ideas come with some significant downsides, but please keep in mind that users posting CSAM present an existential threat to Lemmy (in addition to just being absolutely morally disgusting and actively harmful to the victims of the abuse). If the choice is between having a Lemmy instance with some restrictions, or not having a Lemmy instance at all, then I think the restrictions are the better option.
I also would appreciate your patience in this matter, as all of the long term plans require additional development, and while this is currently a high priority issue for all Lemmy admins, we are all still volunteers and do not have the freedom to dedicate huge amounts of hours to working on new features.
As always, your feedback and thoughts are appreciated, so please feel free to leave a comment if you disagree with any of the plans or if you have any suggestions on how to improve them.
Personally I say just leave hosting of images to dedicated sites for that purpose. Your efforts are better left to dealing with how to render them. That being said, I use to be in charge of managing abuse on a site that has an average of 20 million posts a month (seriously).
The way I essentially defeated these kinds of attacks was with an image scanning service. It scans for anything NSFW and blocks it. Sometimes things would make it through but once an admin flagged it we could use that to block the users IP and account. It’s not cheap but the volume is also not huge yet for lemm.ee so it might not be too bad.
This is my opinion also. Reddit turned to shit around the time they started self-hosting. Imgur only exists because people needed a place to host reddit images.
No, but there's nothing stopping you from using direct links from imgur, in traditional fashion.
It's a little bit convoluted, though. You have to post the image, then hover over and select "Get share links", and then pick the option for BB code (forums). This has the [img] tags at the start and finish, but importantly it has the direct link to the image file. If you use this on lemmy then it will load in the instance, rather than directing to imgur itself.
That solution sounds overly complicated.
Show
That produces
https://i.imgur.com/4K8r2Bo.png
Not yet but I wish there was. I use imgur quite a lot and I like the idea of a fediverse version. Especially with the direction they've gone lately.
For step 6 - are you aware of the tooling the admin at dbzero has built to automate the scanning of images in Lemmy instances? It looks pretty promising.
Yep, I've already tested it and it's one of the options I am considering implementing for lemm.ee as well.
I blocked botart from their instance as some pretty disturbing stuff was added in the last few days.
IMO Lemmy shouldn't have media uploading of any kind. Aside from the CSAM risk, it's unsustainable and I think one of the reasons Reddit went to shit is by getting into the whole image/video/gif hosting.
Dozens of media hosts exist out there, and the mobile/web clients should focus instead on showing remote content better.
There is a privacy and tracking concern with loading images from 3rd-party hosts vs lemm.ee hosting or re-hosting them.
Please please do not implement an invite system.
The success of a forum like this depends on people being able to join and express their thoughts freely. Reddit and digg would never have gotten where they are if they had a closed system.
I almost didn't join lemmy because the first two instances I heard about (lemmy.ml and beehaw) had closed registration. I think I applied and then forgot about it for 2 weeks. Thankfully I saw a post about lemmy on reddit yet again and finally found an open instance.
Don't let the actions of a few scumbags ruin a good thing for everyone. You'll be giving them exactly what they want.
I agree that users should be able to join Lemmy freely, but I think it makes a lot of sense to try and spread users out more between instances - this spreads out the responsibilities between more admins, spreads out the load between more servers and also reduces the chance of a single point of failure for the whole system.
It's clear that there are seriously vile people out there who want to cause huge amounts of damage to Lemmy, and if we have unlimited growth in a few selected instances, then these people only have to target those specific instances for maximum damage.
In a perfect world, none of this would be necessary, but then again, in a perfect world, we wouldn't need a decentralized platform in the first place.
Thanks for responding!
I agree that it's best for the lemmyverse.net if there are many big instances too.
Unfortunately, the concept of the fediverse isn't as easy to understand. The average newcomer (who mostly just wants to consume content and occasionally ask a question or two) starts off by interacting within their instance, and it takes some time to figure out cross-instance communication (there are still posts about this on the nostupidquestions-type communities). For such users, landing on a small instance means they'll poke around the Local active posts, think that "this forum is dead", and never return.
Like reddit, having a large userbase on lemmyverse is important to keep the conversation interesting (see https://i.imgur.com/4tXHAO0.png). Reddit has provided lemmy with a huge shot at success by injecting a large number of users. But if I'm being honest, the conversation on the lemmyverse isn't as diverse and engaging as it is on reddit yet. This isn't self-sustaining yet. I can point to 2 pieces of evidence to support this:
-
Using Voat as a (imperfect) proxy - I don't know if there are official stats of Voat, but the best dataset I've seen for Voat (https://ojs.aaai.org/index.php/ICWSM/article/download/19382/19154/23395) has 16.2M comments in 2.3M submissions from 113k users. Voat was shut down for lack of funding, but even in its heyday it wasn't exactly thriving - many people on Voat were united in their toxicity and it never really got going. Compare these numbers to the lemmyverse which has about 100k active users over the last 6 months. If the fediverse is to grow beyond "that niche forum for nerds", this userbase isn't enough.
-
It's already clear that the number of active users is decreasing - since mid-July, the number of monthly active users has dropped from 70k to 50k. This is expected (bunch of redditors who joined in June, poked around and said hi and left), but it means if the lemmyverse wants to have any chance of succeeding long term, you can't alienate new users now.
The approach I've been advocating since the beginning of lemmy is:
- if you see a user who's interested in lemmy but isn't really tech savvy, just point them to one of the biggest instances. Don't explain what federation is, leave it as a feature to be discovered once they're engaged.
- if you see a user who's interested in the concept of a fediverse and wants to know how it works, explain federation and send them to a smaller instance.
The way federation works now, it's still disadvantageous to be on a smaller instance (discoverability of new communities is harder, syncing posts/comments isn't always fast, it's hard to know which community is more active. Many of these can be fixed with changes to activitypub and lemmy protocol, but in the meantime, sending casual users to small instances means they'll likely never return.
So to sum up, I think there should be an avenue for casual users to join the biggest instances, even as we encourage people to move to smaller ones (either targeting those who are more tech savvy, or those who have already been on Lemmy long enough to know how it works - I myself was on Lemmy.world and switched to this "smaller" instance).
Anyway, you're the admins here and I have no say over what you eventually do. I'm just hoping you'll consider the practical realities of user behavior - everyone wants what's best for the fediverse in the long term.
discoverability of new communities is harder
https://github.com/Fmstrat/lcs
syncing posts/comments isn’t always fast
My experience is the opposite, but that may be instance dependant
it’s hard to know which community is more active
Active users stats are the same on every instance for communities
-
If I may, lemm.ee is now the second biggest instance. Redirecting people to register on local instances (feddit.country) or generalist ones (reddthat.com, Lemmy.today, discuss.online etc.) couldebe reasonable to make those ones grow as well.
I agree that there should be a clear lists of instances open for registrations, but that probably needs to wait for the dust to settle a bit beforehand
While I understand your concerns, this instance has gotten a fair bit larger and will start to suffer the same issues that lemmy.world does if registrations aren't curbed. It can't grow infinitely. That just isn't feasible for one server. Having closed registrations on lemm.ee doesn't stop anyone from signing up on different instances. A solution might be to temporarily limit registration here in some way, and for the devs and instance admins to find a better way of helping new users choose an instance. The initial sign up process was confusing, and could be streamlined to make it easier for people to choose an instance. In the long term, enhancing the way federation works so users who do sign up on smaller/newer instances don't need to be lemmy savvy to find content would also help alleviate that type of issue.
Forums have existed on the internet forever and and have already dealt with this thousands of times previously. You don't need to overthink it or reinvent the wheel. It didn't stop forums existing very comfortably in the past and isn't an issue that should be that different to deal with today.
Simply limit image uploads to a certain account age threshold and karma threshold and you will eliminate 99% of the ability to abuse this.
Forums have existed on the internet forever and and have already dealt with this thousands of times previously
The main difference is that forums aren't federated. On Lemmy you not only need to keep in check internal users, but also external instances, and as everyone can host one, federation ads extra complexity
Not really. We've had forums that literally allow you to post to them without even signing up with an account. Without being a "user" at all. This isn't about "checking" anyone, it's simply about limiting its ability to be used as a troll tool below the point at which it becomes too tedious to bother. At that point you have eliminated 99% of it.
This CSAM poster is 1 single person among hundreds of thousands. Making it too tedious to perform eliminates them along with the problem entirely.
I disagree with karma as a concept, but I agree that there should be restrictions of this kind in place. It's not user friendly, but if it minimizes the chance of someone uploading sick stuff on Lemmy then I support it.
Also, maybe some kind of autohide/minimise posts that have been down voted to a certain threshold.
Let users contribute to a central database similar to Sponsoblock. Posts flag by a Power/Trusted users would be immediately hidden pending review.
thank you for your work sunaurus, and i'm sorry you had to sort through this
(particularly annoying though, as i never got around to adding a user banner; and i had one in mind as well. i wish there was some way to externally host avatars and banners)
I'm going to be a part of an invite only community?! Of course, given the circumstances, this is pretty fucked. But I feel kinda fancy right now.
Thanks for all you do on lemm.ee
I left Twitter before musk, when the security chief said that they know they have CP but they were doing nothing.
I can forgive a measure that doesn't work as expected or at 100% but not the inactivity.
Therefore I'm agree with any measure you think it can work despite any inconveniences for me.
Sorry for any misspelled or wrong word, English isn't my main language
Regards and thanks for all your efforts.
Your English is flawless and your sentiment is echoed. The last thing we should do is to ignore the problem.
Could you post a guide on disabling the local image cache? I compile from scratch so I’m not afraid of making changes in the code, I just don’t really know rust. I shut down my personal instance and this would allow me to turn it back on.
This should do the trick: https://github.com/LemmyNet/lemmy/commit/c74b92e5fd3b6e4b815cb76768bb8b042784ef6c
(I am fairly certain that this is the only way external post images get saved locally, but if I find additional ways later, I will update this comment)
This has been a great instance since day one, and it's good to see you once again being so proactive. Thank you for the update!
There are downsides with all kinds of moderation, but ultimately most of us accept that the internet can't function as a true free-for-all. Absolutely in support of whatever you feel is necessary to keep the server safe, but please watch out for yourself too and make sure you're asking for help where needed.
p.s. anyone reading this who doesn't donate to the server yet, here's a reminder that that's a thing you can do.
Got to be honest, having an invite based system and locking certian features behind age of accounts, karma, etc seems like the opposite of the freedom everyone promised me the Fediverse represented when we moved over.
I personally don't really care about images and would prefer image uploads just stay deactivated and we operate as a text only forum but with open membership.
Leaving image uploads completely disabled would also be an option to fight this particular type of attack, but there are also other issues with open registrations. For example, while our sign-up captcha seems to be preventing automated registrations, we are still having to ban advertiser accounts almost daily. I think an invite system would really help to reduce sign-ups by any kind of users intending to abuse the system.
I'm all for an invite-based system, although we will need some way of combating 'invite trees', where one bad actor invites several others, who subsequently invite an exponentially increasing number. A reasonable delay on the invite allowance would go a long way, I think.
I have to say that an invite based signup system makes my toenails curl backwards. IMO this will let instances die out slowly. I didn't know anyone using lemmy and just stumbled upon it. ppl like me wont ever be able to join an instance if it is invite only.
Don't misunderstand me: I do understand how critical it is for the operators of instances to protect themselves. Lemmy is a rather young project and still needs better admin tools. However, there are some good discussions happening on GitHub. Untill the operators and admins have the tooling to protect themselves, I see disabling img upload as preferable. It also took reddit some time to allow uploading images, instead of linking them.
I 100% agree! An invite-based system means that a new user has to find some way of contacting someone in order to request an invite. I think that only allowing X posts per day for e.g. the first week or 2 for new accounts would be a way to combat companies and spammers. Not allowing images or limiting image posts for new accounts, and using automated CSAM detection methods, which I understand are in the works, seems to be a good way to combat that problem.
I very much agree, invite-only systems are a bad idea for this reason.
This is something that many admins and mods have been discussing for a while now, and I believe it would be an important feature for lemm.ee as well. Essentially, I would like to limit certain activities to users which meet specific requirements (maybe account age, amount of comments, etc). These activities might include things like image uploads, community creation, perhaps even private messages.
Sounds like the old karma requirements some reddit subs had. While I'm not against that, it would restrict locally registered users more so than others who are posting on lemm.ee communities when their host instance has no such system in place. I'm aware that if they post images those would be uploaded to their home instance and linked here with the patch you mentioned above, but the downside is that local users might feel inconvenienced more so than others. Not saying it's a bad idea though, if we are thinking from a "protect lemm.ee" angle first and foremost.
Automated ML based NSFW scanning for all uploaded images
You might want to reach out to the dev of Sync for Lemmy, ljdawson on !syncforlemmy@lemmy.world, he just implemented an anti-NSFW upload feature in the app to do his part. Essentially, Sync users currently can't post any kind of porn. While I don't think that the CP spammers were using his particular app, or any app to begin with, I do think it's a neat feature to have, but would make much more sense to run server-side.
he just implemented an anti-NSFW upload feature in the app to do his part. Essentially, Sync users currently can’t post any kind of porn
but what about normal, legal, NSFW material?
Not allowed on lemm.ee in the first place. Well, you can see NSFW posts and subscribe to everything on lemmynsfw.com but you're not supposed to post any porn from a lemm.ee account.
Policing NSFW is a whole can of worms, it makes sense to leave it to specialised instances. They can nuke political drama from orbit, we can nuke nudity from orbit, both saving mod bandwidth to do the other thing right.
Thanks for keeping the community updated and for all the work you put into maintaining it!
A karma system is sounding pretty good right now... /me lifts shield and ducks
Even if it's just a a limited tiered system with numbers to obsess about. Level - 1 browsing rights. Graduate to level 2 after 5 days and total of greater than 30minutes of logged in activity
Level - 2 commenting rights. Limited to 10 comments daily for 5 days.
Graduate after at least 3 comments, total upvote count >+3, and 5days.
Level 3 - posting rights. Limited to 3 posts daily for 5 days. Unlimited commenting.
Graduate after 5d and total upvote count >50
Level 4 — image posting rights. 10 images per day max
Graduate after 2 weeks and total upvote count >100
Level 5 - you've made it, everyone is equal here. Entry level users are still enjoying and growing into the community. No need to be a tool about trying to get more karma / points and number of bots / temp accounts / total losers should be minimal by this screening level.
Unlike the features mentioned in the OP -- all of which I support, though I regret 4's necessity -- I think this one would actually be harmful to the existing userbase because karma scores encourage pointless attention-seeking behavior, as Reddit demonstrates.
I don't see how that would happen if everyone just capped out at karma=5 within a few weeks
A karma system is sounding pretty good right now
lemmy's code already does it. person_aggregates keeps track of post_score and comment_score. It just isn't displayed on lemmy-ui. A bot or new code can look at these values.
I hope there is another option besides just deleting images indiscriminately. I run several comic strip communities and it would be a shame to lose all the posts and work I've put in.
What about implementing Imgur or something similar, assuming they scan for CSAM on their end. For example I often use the Lemmy iOS app and I noticed that all my image uploads using the app are through Imgur.
@TWeaK is correct, I am only deleting our copies of images which are already hosted on other instances.
As for imgur (or any other external image host), such images have always worked on lemm.ee. For example, this is hosted on imgur:
Show
In addition to using external images in comments, you are also able to submit posts with imgur images, and they will get embedded directly into the Lemmy UI.
You wouldn't lose the posts you've made, rather the posts you've made will be hosted from one instance, rather than all of them.
You're a lemm.ee user, if you upload to a lemm.ee community nothing will change.
If you upload to another community, then normally you're post would be uploaded to lemm.ee. This would then be federated, and users from other instances would load the same content, but it would be delivered by their own instance.
The change refers to things beings hosted only in your host instance. Thus, a lemm.ee user may load content from a lemmy.world server more often. Normally, lemm.ee would copy the content to its own servers and direct its users to that, but now everything will go to the host instance.
The only thing I'm not sure about is who is the host instance? My understanding is that the host instance is that which the user belongs to. Thus, if a lemm.ee user posts to a community in lemmy.world, techincally the federated host instance is still lemm.ee - it's about the user, not the community. But with all this I'm not sure.
Side note, but thanks for posting all the comics. I see them all the time.
