The Twitter thread is by the author of the Rolling Stone article.

Spoiler

NEW from me: The true crime fandom has just gotten a lot grosser. TikTok creators are using AI to post videos of real-life child murder victims telling the stories of their gruesome deaths. Technically, deepfake videos portraying child murder victims are in violation of TikTok's synthetic media policy, which prohibits deepfakes of private individuals, a spokesperson confirmed. But some of these videos are getting millions of views regardless.

Obviously, this is incredibly ethically dubious. "Imagine being the parent or relative of one of these kids in these AI videos," one expert says. "You go online and here's your deceased child, going into very gory detail about what happened to them." Like many in the true crime community, however, creators defend these videos by claiming they are spreading awareness for educational purposes and not disrespecting the families.

It also raises the question — exactly where does this end? Are people going to start posting AI recreations of grisly murders soon, in the name of "spreading awareness"? Honestly, one of the weirdest things about this: the creators try to skirt community guidelines by using diff photos for the victims. so often the videos feature kids of an entirely different race (like Junko Furuta, who is portrayed often by a white girl).

Tweet

  • save_vs_death [they/them]
    ·
    2 years ago

    the only conclusion i can make is that nonces are using AI to churn out CP, 100%, they're probably up to a zettabyte by now

    • Changeling [it/its]
      ·
      2 years ago

      LEOs have had massive databases of known CP for decades, as have internet utilities like Cloudflare. It’s probably one of the most consolidated forms of media in human history simply by the nature that the more you have of it, the easier it is to combat and nearly everyone wants to combat it categorically.

      So the idea that there aren’t any closet pedos in any of the positions which have access to these databases and could get copies is preposterous. And given how mature and vast their underground networks are online, those databases would have buyers.

          • GreenTeaRedFlag [any]
            ·
            2 years ago

            I see. Follow up, why would having more of it make it easier to fight?

            • The_Walkening [none/use name]
              ·
              2 years ago

              You can compare video files found on a computer or phone to what you have a database of - that way you don't actually need to view it to verify it's CSAM.

                • Changeling [it/its]
                  ·
                  2 years ago

                  In terms of AI training, increasing the accuracy of the output is often a matter of increasing the order of magnitude of your training data. So old CP can be used to more accurately flag potential untracked CP, which can help construct social maps of active CP peddlers.

    • ssjmarx [he/him]
      ·
      2 years ago

      I'm certain that there has been deepfake CP since before the current "AI" image generation came about. It's actually specified in the law as illegal to make/possess, so it's old enough for politicians to know about it at least.