The Twitter thread is by the author of the Rolling Stone article.

Spoiler

NEW from me: The true crime fandom has just gotten a lot grosser. TikTok creators are using AI to post videos of real-life child murder victims telling the stories of their gruesome deaths. Technically, deepfake videos portraying child murder victims are in violation of TikTok's synthetic media policy, which prohibits deepfakes of private individuals, a spokesperson confirmed. But some of these videos are getting millions of views regardless.

Obviously, this is incredibly ethically dubious. "Imagine being the parent or relative of one of these kids in these AI videos," one expert says. "You go online and here's your deceased child, going into very gory detail about what happened to them." Like many in the true crime community, however, creators defend these videos by claiming they are spreading awareness for educational purposes and not disrespecting the families.

It also raises the question — exactly where does this end? Are people going to start posting AI recreations of grisly murders soon, in the name of "spreading awareness"? Honestly, one of the weirdest things about this: the creators try to skirt community guidelines by using diff photos for the victims. so often the videos feature kids of an entirely different race (like Junko Furuta, who is portrayed often by a white girl).

Tweet

  • GreenTeaRedFlag [any]
    ·
    2 years ago

    I see. Follow up, why would having more of it make it easier to fight?

    • The_Walkening [none/use name]
      ·
      2 years ago

      You can compare video files found on a computer or phone to what you have a database of - that way you don't actually need to view it to verify it's CSAM.

        • Changeling [it/its]
          ·
          2 years ago

          In terms of AI training, increasing the accuracy of the output is often a matter of increasing the order of magnitude of your training data. So old CP can be used to more accurately flag potential untracked CP, which can help construct social maps of active CP peddlers.