Chris Paukert on Twitter
twitter.com
This is terrifying: a marketer pinged me for quote permission — common for a journalist. Only I didn’t recognize the text as something I’d ever said. Pumping it into Google returned nothing.I asked where he got it: ChatGPT. AI fabricated a quote and attributed it to me. 🧵 pic.twitter.com/FbkKoDNwXH— Chris Paukert (@CPAutoScribe) May 4, 2023

This is going to wreck society even more.

Please, for the love of Marx, do not take ChatGPT at its word on anything. It has no intelligence. No ability to sort the truth from fiction. It is just a very advanced chat bot that knows how to string words together in a sentence and paragraph.

DO NOT BELIEVE IT. IT DOES NOT CITE SOURCES.

spoiler

I feel like my HS English/Science teacher begging kids to not use Wikipedia, right now.

But even Wikipedia is better than ChatGPT because. Wikipedia. Cites. Sources.

    • supermangoman [he/him, they/them]
      ·
      2 years ago

      Showing a correlation between board states and internal representations on in-distribution data doesn’t disprove that it’s surface statistics at all.

      The article doesn't make such a bold claim. It presents its goal as "exploring" the question, so not sure why the redditor started off with that.

      If they had done an experiment where they change the distribution of the data, like a larger board, or restrict training to a part of the board, and it still works, that would show something.

      Why?

      They then “intervene” on all the later layers to get the result they wanted, proving that “intervening” on a single intermediate representation isn’t enough. It seems like this is the core claim by the post - that the author "led the witness" so to speak to get the desired outcome. Despite using the word "graident," the redditor does not really explain this, but could be true. It's definitely worth going through the appendices.

      Appreciate the (semi-anonymous?) critique regardless.