I used to think that even though Apple's bad, at least they're okay on the privacy front. But what the fuck?
They wanna be able to detect every image I take for the possibility of it being "illegal"? Fuck you.
Is Ubuntu good? And LineageOS for phone?
Edit - At least r/apple seems to be against it If there's one positive thing I can say about reddit, it's that those nerds are usually all in on privacy stuff.
Not publicly available, no, but I imagine the images are kept somewhere, so as to allow them to be hashed with new algorithms as they're developed. Maybe Apple just supplied the algorithm and got back a set of hashes?
Anyway, the excerpt I posted above makes it very clear that the neural net step comes first, and then the result from the neural net is hashed. So there's no way to just use the old hash database for the system they're describing.
Hmm... okay, now that I read it again I think I understand it more. It sounds like this particular AI setup is for making it able to see past different types of obfuscation than a normal perceptual hash algorithm. Honestly, given that the normal algorithm in use now is Microsoft's PhotoDNA, it sounds like they primarily made this mostly because they don't want to implement something made by Microsoft if they can avoid it, even at great expense (and yes, you're right that they probably made a new hash DB for their algorithm -- apparently the hash database is less old than I thought, 2015). As far as I can tell from reading this technical summary it sounds like the neural net itself is used to determine what parts of the image are important enough to be factored into the hash, so not necessarily the type of thing people are used to tricking AIs with.
As far as fooling the neural net goes, I don't think there is any real way to test since there isn't even a way for the user to know whether an image was flagged. It sends a voucher with the result of a test off to Apple, but it is impossible for Apple to decrypt these until a certain number are uploaded. Then there's a manual review before any user-visible action happens to the account, so there goes any plans to make something innocuous that triggers it. As far as fooling it into a false positive goes, whoever wants to attempt that sure has their work cut out for them. Unless someone can reverse-engineer a neural network to make a hash collision with a database they cannot see the results of a test against, I think the chances of this happening are slim. Because assuming that Apple is not outright lying about parts of this (in which case why mention it at all), you'd have to send off several compromised images that look close enough to the real thing that they pass manual review. Then it would get to NCMEC who would check it against their images, and find that they are fake, because they have the real image to compare against. So nothing would really come of this.
But I've already heard half a dozen variants of "what if see see pee makes apple scan phones for pictures of tiananmen square tank man" in less than a week from this news breaking, so I think the brain damage from this is already done to the general population.
Well given that it happens client-side, the software that does the actual hash has to be loaded onto the phone, right? So the algorithm itself can be retrieved with some reverse engineering, and I doubt nefarious actors would have much trouble getting hold of CSAM images that are likely to be in the database. Which means that it's certainly possible to game the system. That said, it would probably take a state-level actor to actually do it, but that doesn't mean it couldn't happen. You're right that it wouldn't amount to much of anything, though, so I doubt anyone would bother.
Anyway, your point about privacy in the cloud is quite salient, I think. Personally I don't have an Apple phone, nor do I use cloud storage services, and I sure as hell don't have any CSAM, so I'm not really concerned about it. In fact, I was originally making a point similar to yours, and had to backtrack somewhat when I actually looked into how it all works.