Hexbear was a victim of a targeted XSS attack similar to the attack many other Lemmy instances have seen.
The account that first leveraged the attack was registered on 2023-07-10 at 03:58 UTC, the fix for the vulnerability was applied by around 04:35 UTC. This leaves a ~40 minute window in which anyone browsing the site could have had their account hijacked.
The attacker was able to act (post, comment, DM) as the account they hijacked. They will also have been able to view/use the compromised account's settings page. This means they will have been able to see users' email addresses. Some accounts that were compromised were temporarily banned, these bans have now been lifted.
If you were using the site during the above time window, please double check your account settings to see if anything was changed.
Passwords were not stolen, JWTs were. We have just invalidated all old JWTs so the attacker no longer has access to the hijacked accounts (this is why all users have been logged out).
Just to let you know, anything cringy I post from now on is actually just because I got hijacked
This is going into my next ban appeal
The hackers made me do it
Excited for the torrent of people that are locked out of their account because they don't use a password manager.
use a password manager. it's hygiene for your digital life. scrub with soap.
Was here the whole time.
Did not even notice, I figured it simply hadn't worked or hexbear hadn't been targeted because it's not federated or in lists. Handled extremely smoothly compared to other instances.
This actually gives at least some idea about how they went about certain things. Whatever list of lemmy sites they used had to include Hexbear. They didn't just use federated instance lists.
One of our devs was online at the time and patched it very quickly, and seemingly no admin accounts were hijacked, so all they could do was normal wrecker shit.
We have the finest devs, folks. Many people are saying this
How international are the devs and admins? I don’t expect them to be on 24/7 but it would be good if the regions were diversified so someone would be notified while the other is sleeping
judging by the response time on this issue I think pretty diversified. I'd be surprised if there was anyone west of the US or east of central europe though just based on site demographics
This is an important milestone, as everyone was finally logged out for a reason other than something I did
@Admins: please do not assume the current lemmy bug report is exhaustive; do not simply select for "%onload%" or something in the DB. "%onclick%" would work too, for example (even if it'd require the target to click the emoji).
I'd quickly, visually check any comment containing quotes in the affected window (and even a bit outside of it)
Also this is exactly the kind of fucking thing that wouldn't happen were we not required to use JS to access lemmy; though I shouldn't be surprised given the frontend at least IIRC is fucking made out of the abomination that is nodejs. Deploying that thing cleanly must be painful, too
At any rate kudos to the admins for reacting so quickly
We didn't just remove the comments/DMs etc from the DB, we actually fixed the vulnerability so even if other malicious scripts were used they won't work anymore
Agreed, about JS 😅
I should've realized since I was able to post a custom emoji (or, you know, had I read your initial post properly 🤦)
Thanks again for all you do !
Also this is exactly the kind of fucking thing that wouldn't happen were we not required to use JS to access lemmy; though I shouldn't be surprised given the frontend at least IIRC is fucking made out of the abomination that is nodejs.
Not only that, but the framework we use is infernojs. Imagine react, but it doesn't have hooks, has barely any documentation, and has virtually no compatible UI libraries or tooling. It's significantly more performant and has a smaller bundle size than react, but the negatives outweigh the positives.
In particular, there is no isomorphic framework for inferno like next for react, nuxt for vue, sveltkit for svelte, etc. To get around this, there's a filthy kludge where we take an object of isomorphic data and assign it to a property on
windowin a script tag. This is both error prone and insecure, but also required for the UI to work without needing to generate the whole page on the client side. If you wantb to see this for yourself, open your browser's devtools and look for a script tag towards the top.
for the Leptos WASM rewrite.I looked through the recent PRs and I'm still confused about how non-admin inputted data is being rendered to the user as raw HTML. I suppose that's just a problem with react's SSR and/or why we have non-http-only cookies?
This is the bug report you want to read (well this one too obviously but I assume you saw that one, and it was the former that made me manage a working payload in my test env, not this one - they reasonably censured the latter in case of copycats); basically IIRC: faulty escaping of control characters (and then html/js) in the custom emoji syntax. With a set of functions also shared by the legal and sidebar customizable-by-mods parts (but really the custom emoji thing is the main route to wholesale exploitation obviously).
Taking a step back, remarkable real-life-cyber-exercise for the lemmy community, by the way: it was all patched in like 48hours max for basically all instances I've cared to look at. Believe me from experience: critical (as in defense-related) capitalist entities wish they had that kind of turnaround on cyber issues to be honest. Also, a lucky break too of course; seems only to be one attacker, with a shitty domain name for the payload on top of it. Could have been noticeably worse (like a wide variety of DNS used with varied payloads and triggers + automated admin JWT token stealing + replication through the other lemmy customizable parts through admin accounts + mass-fetching of user emails and publication of the same; you know, the horror scenario I could see an intelligence agency getting hold of such a flaw would go for).
More and more I'm having my "write-a-no-javascript-needed-lemmy-frontend-using-the-same-neat-css-tricks-as-darknet-markets-for-usability" envy raising; writing it in something sane, too, like python+flask. I pray for the free time to do it.
I'd really like a reddit-like UI, much like the one we're currently using actually; just without JS.
https://mlmym.org/hexbear.net/ this is that I think
Not like the one we are using though. just an exact clone of old reddit lol
OK that's... not great but surprisingly close, and I'm very surprised it exists in the first place. Thanks !
yeah lol. it has a dark mode even! and we could self host it in single instance mode. browsing at least works without js, didn't want to give them my login info to see if anything else did
03:58 UTC, the fix for the vulnerability was applied by around 04:35 UTC.
Impressive job.
But any idea who did it and why? What were they doing with hijacked accounts?
So other instances were attacked already? Damn. I wanted to use it myself on one of the liberal shitholes to replace every picture with a PPB.
There is suspicion it came from a widely defederated lemmy called exploding-heads I heard. Since when they compromised lemmy.world they re-federated that instance
Glad the damage has been minimal. Once the login issue is resolved we should make a pinned thread so everybody who was logged out can be made aware that they can log back in, as I am sure many people now are experiencing the same login issues that began during the site updates of the past few weeks. (this account is the 4th I have made in that time, all others can not log in on any device or browser)
I've been seeing these messages periodically and don't know how to handle this issue as we haven't been able to replicate the issue locally at all. Would you (or anyone else experiencing login issues) be willing to hop on Matrix or DM, or whatever works for you to try some troubleshooting steps with me or a member of the dev team?
I believe our prime suspect is a cookie that's not getting cleared, but there are some other things I'd love to check.
Hi, when both manually typing and also pasting (not very safe i know but i had to know if it was a user error when inputting it) my new password when attempting to log into my account from a different browser (caches cleared, history cleared in this new browser) after switching my original password after the forced logout, I also experienced the spinning bear.
I went back to the other browser where I'm still logged in to change passwords a second time, and after pasting my recently changed password, the new password change went through and I was then able login from another browser. I wouldn't have been able to switch my second password another time if I had an incorrectly recorded password, so I believe for quite a few users that are unable to login (including one friend who I communicate off-site with), they are experiencing issues with their account logins that doesn't originate with user input errors or with uncleared caches. Very glad to have my account logged in to at least one other browser else I'd be perma-locked out of my account due to the dreaded spinning bear, prompted me to create a burner email to prevent a logout + spinning bear situation in the future.
E: Please let me know if I can assist in some way with this issue and thanks again for you and other devs and admins' work on our beautiful bear site.
hmmm if it's a failed password change there likely isn't much the admins can do.
input of the old password led to the spinning bear earlier on a non-logged in browser, while the same old password was successful when confirming a new password change in settings on a different browser where I am logged on to the website, it's not the old password itself that's an issue but something that has to be with the process of logging in, else I would not have been able to change my password on the browser I'm still logged in on
ohhhh gotcha
if you can still reproduce the problem on that other browser I'd check for things like cookies and cached service workers and stuff... (if it's firefox, ctrl-shift-e and then go to the storage tab to look at cookies. Things like the domain, expiration date, and settings like httponly, secure, samesite should be innocuous to share but potentially useful to infer if it's an old stuck cookie, or using the wrong domain, etc)
Yes I actually need to get on the Matrix anyway since I am a mod as well! Send me a DM with what to do
shit like this is why i only register with burner emails (or, in the case of this site, no email) if possible.
Also I always use 100 character long randomized passwords that are different on each site. Overkill? Yes. would rather be overkill
There's still a bunch of sites I run into that limit password lengths. I accept there has to be some limit, but if you're just hashing it and comparing it to a known value anyway it's really negligible compared to SMS 2FA rates, checking FIDO message signatures, or using some like SQRL/Google's new thing.
Phishing sites usually don't limit password field length so I ctrl+v the entire bee movie script in there a few dozen times then hit enter over and over until I get bored when I find one.
I just want to say that I saw all the shock content, and.....yeah. Goddamn.
Props to the mods and devs for getting on it so fast last night/this morning.
to our big beautiful admin team for the quick fix.
that goon and their exploit.