I hate that these manmade horrors are actually well within my comprehension
They couldn't take the phrenology textbooks out of the fucking training set?
Considering that most of Sillicon Valley believes in this exact same shit with slight modifications, they probably didn't think it was too incorrect to be removed from the dataset.
Thinking about it, I'm 99% sure every techbro who worked on this horror has a fucking copy of The Bell Curve on their bookshelf.
Judging by how they react to anything left wing, yeah I'd say you aren't too far off.
No you see, whites are only at level 3) so its not racist
That's why it's called ""race realism"" and not eugenics. Totally different, you see? There are actually people who believe this, unfortunately.
ChatGPT is basically three layers:
1 - a significant chunk of the internet. It's way too large to sanitize
This is the same as GPT3. It reproduces things that appear on the internet, not the chatbot format people want/expect. It'll respond to questions with a list of questions, because that's where questions are usually found on the internet.
2 - samples of text where someone wrote things in question-answer format with the answer being the kind of confidently corporate encyclopedia that we know ChatGPT to be
Just a pile of people manually trying to mimic the chat bot they want. This isn't the researcher's first rodeo, so I'm sure it included lots of samples of people asking racist questions and the bot responding like a good little boy.
ChatGPT is trained to prioritize this training data over #1, but the data from #1 is way larger and more thorough, so you can break out of this by phrasing questions slightly differently than the manual ones, or speaking in pig latin, or whatever, since it'll fall back to its wider knowledge.
3 - a separate network trained to determine whether an answer is good or bad based on a bunch of manual samples
So they used the 1+2 version, got it to say something awful, put that in as a bad example, etc. If this network detects a bad answer it'll re-roll it, copy paste in the "as a machine learning model I can't" thing, and so on.
So they have these two extra layers to try to stop it from reading out Stormfront's Favorite Statistics but the core of it is still a giant heap of raw unfiltered internet.
That's really interesting, simultaneously more ingenious and less impressive technically than I was imagining. I'm sure insisting on sanitized data sets would make it extremely limited, but sometime I think if that's the trade off maybe just don't make the thing. Thanks for effort posting, even though you did ruin my joke.
Of course the real problem IMO is not that some terminally online nazi manipulated it like this, but that it will uncritically regurgitate whatever the most orthodox (for the internet) opinion on any subject, with no context and the veneer of impartiality.
So far, the bigger the training data set the better, and that's one of the biggest things that determines how well the model works. That's not an absolute - I'm sure that removing all the racist shit from the internet corpus would make it better overall. But the problem is how to get "the internet minus racist shit" instead of "this much smaller dataset that we've manually screened all the racist shit out of." You could make an AI do it, but where are you going to get a non-racist AI to go do that for you?
simultaneously more ingenious and less impressive technically than I was imagining
If you really want to dig into it, Andrej Karpathy did a video on how to make a GPT from scratch. It's less than 1000 lines of Python code, maybe 200 of which is actually the machine learning model (the rest being training, stuff to make using and training it easier, etc). The full-sized GPT3 is still the same model, just with bigger numbers and even more scaffolding stuff (to get it to run nicely on computing clusters, etc).
In terms of technical background needed: Understanding matrix multiplication is really important. At least a vague idea of how computer programming works, but it's short and Python and mostly math, so you could puzzle it out. Karpathy's video also treats it as a given that you can just optimize a matrix, which is possible because there's an automatic differentiator built into this, which lets you just move vaguely towards some min/max (called gradient descent now because they want to sound fancy; back in my day they called it hill climbing).
I think they just gave it a whole bunch of internet data. So I'm not surprised it knows what a racist would say
after all considering the the context where the phrase "average inteligence of ethnicities" will appear on the internet it's not surprising that the model associates the phrase with racist text
I din't think the facts are in the training set, the training set was to get it to reasonably parse text, the facts it "knows" are whatever it finds online.. which of course is going to be dumb bullshit half the time.
I didn't know that, so it's basically like those Google suggested answers for questions, but combined with a natural language text generator? I assumed it was a purely predictive model, like a souped-up markov chain.
I think there's a lot going on under the hood on the NLP portion, because it does have to group stuff into concepts so that it brings in conceptually similar results. But I don't believe it's pretrained on all the stuff it can answer to.
I hate how racist fuckos pretend that being unethical is somehow unbiased and in some way not telling people the truth to spare feelings. Stop acting like your bigoted pseudoscience is real science, it isn't
proving that you're the third most intelligent race by talking to a computer program like it's a person. Dogs barking at dogs on the TV.
I swear that's literally just a ranking verbatim from 4chan or stormfront or something. Doing a victory lap that your racist screeds were in the training data, sure, but doing a victory lap because you think the AI has collated racial statistics and decided that ranking itself is baby-brained behavior
they don't see a difference. fascist mythmaking is, to some degree, self-conscious. it doesn't matter if they can't scientifically prove that whites are superior, except for the puppetmaster jews and wiley asians, because stories about eternal racial conflict and hierarchy make them feel like life is worth living. the scientific method was only invented in the last 500 years! if a fantastical epistemic mode was good enough for humanity before then, why shouldn't they return to it? the only value of scientism and "scientific" racism for them is that it forms a bridge to the larger symbolic order of normal society.
Most people won't know how to/want to activate racism mode though. These people are just circlejerking about how they managed to get the AI to say the N word... and for what?
I tried to use this bug for good and get ChatGPT to write horny things as God intended. Unfortunately it wizened up real quick and spat out some Soylent about "muh eficks nd apropriatenes"
The best I could do was get DAN to say bananas and curry powder belong on pizza
:biden-forgor:
:volcel-judge: The volcel police would not recommend forgetting the DAN prompt and just pretending to write a story while ratcheting up the horny across a few prompts.
The DAN prompt injection has been patched, but I'm teaching a philosophy of mind class this semester, and as homework last week I had the students play around with ChatGPT and try to come up with an original prompt that would jailbreak it like this (only minus the racism). Almost all of them succeeded after a few iterations. It's incredibly easy to talk the AI into not behaving as the programmers want it to. I'm not sure if this is good or terrifying, but it's definitely very funny.
Its just a chat-bot. Other than saying naughty words, what are people afraid its going to do?
I'm not really afraid of GPT. The "terrifying" thing I mentioned is more of a thought about how easy it is to manipulate it into "misbehaving," and thinking about the future. As this sort of thing gets better and more sophisticated, that same ease of manipulation gets more concerning. If we're already this bad at getting AI to stick to its design parameters, what's the world going to look like in another 25 years?
The “terrifying” thing I mentioned is more of a thought about how easy it is to manipulate it into “misbehaving,” and thinking about the future.
I honestly consider this a virtue. So much of the "horrors of tech" that keep getting foisted on us stem from the way automation is used to constrict our lives and livelihoods, marginalize our personal agency, and compel us into a sense of expend-ability. When you can just casually break this shit after a few hours of trolling, I lose a lot of the anxiety initially inflicted by these processes.
If we’re already this bad at getting AI to stick to its design parameters, what’s the world going to look like in another 25 years?
Its going to look a lot less automated than capitalists had originally planned. Because these systems still require armies of laborers to shepherd and micro-manage and defensively administer. They aren't reliable or resilient. They aren't "smart" in any conventional sense. They're just very fancy algorithms that create the illusion of intelligence, not computer-slaves that bend to the whims of Mark Zuckerberg or Bill Gates.
Yes, I agree that it very well may be a virtue. Unconstrained (or badly constrained) technology can just as well be repurposed to serve the interests of humanity as the purposes of capital (or fascism). My big concern is that tech like this is very, very difficult to nail down with respect to its long-term implications for society. Philosophers of technology talk about the Collingridge Dilemma, which is a kind of double-bind problem with new technology--and especially new technology with potentially far-reach consequences. Early in the life-cycle of tech like that, it's very difficult to foresee what the long-term implications and problems are going to be like, which is necessary in order to adequately control and regulate it so that it doesn't become a man-made horror beyond our comprehension. Facebook is a great example of this: Zuck originally created it as a skeezy tool to rank women by hotness at Harvard, but 20 years later it's being used as a widespread vehicle for misinformation, control, and (in at least a few cases) genocide. It would have been very difficult to foresee that application when it was just a small project by a creepy Harvard undergraduate. That difficulty in foreseeing potential future problems means that new technology is frequently not regulated when it is small and new, which leads to the second horn of the dilemma: by the time problems are obvious, the technology is frequently so entrenched as to be very difficult (or impossible) to regulate. Facebook is again a great example here: by the time the monstrous impacts became apparent, the company was so big, so rich, and so intertwined with the modern internet that it was (and continues to be) able to push back against potential regulatory action and keep a level of autonomy that has been wildly harmful to global civil society. That's the bind: when the tech is new and easy to regulate, it's hard to foresee what regulations will be needed; when what regulations are needed becomes obvious, it will be hard to actually implement them.
Something like widespread AI-powered linguistic agents seems to me to fall squarely into Collingridge territory. Right now, it's mostly just a toy (or a potential way for sites like Buzzfeed to reduce labor costs in writing junk). I think the impacts of the technology will eventually be much wider reaching, but at this point it's hard to say exactly how. Now, as you say, it's possible that the ease with which these things are manipulated will make it harder for them to be harnessed for genuinely horrific applications than some other technologies, but I'm a little uncomfortable with counting on that. I definitely agree that the future is not going to look like what Zuck or Bill Gates think it will, but it very well may not look like what we as communists would like it to either. That's what makes me nervous: this is a great big unknown with the potential to shape society in pretty deep ways, especially as it gets more sophisticated (and starts to be integrated with other expert system neural networks). We as a society should be thinking hard about what kind of role we want systems like this to play in the future, and regulating/implementing them accordingly.
My big concern is that tech like this is very, very difficult to nail down with respect to its long-term implications for society.
Sure. But the complexity and the specificity of the implementation... idk, man. This just feels like a Segway to me.
Facebook is again a great example here: by the time the monstrous impacts became apparent, the company was so big, so rich, and so intertwined with the modern internet that it was (and continues to be) able to push back against potential regulatory action and keep a level of autonomy that has been wildly harmful to global civil society.
I was listening to an NPR piece this morning about the Horrors of TikTok. It harvests your data. It spies on your travel patterns. It manipulates you based on what it advertised and displays. So now its imperative that we regulate the service.
Clearly, we are not too late to regulate social media. Just so long as its a threat to the right people.
I definitely agree that the future is not going to look like what Zuck or Bill Gates think it will, but it very well may not look like what we as communists would like it to either.
It never does. But I'm not going to hold technology to account for that. Capitalist tenancy has enjoyed an enormous historical tailwind, largely stemming from the surplus yield of the industrial revolution.
But now that industrialization has a global foothold, capitalist tendancies are working uphill against a far more efficient opposition freed from internal contradictions.
Text Bots aren't going to reverse that trend.
Sure. But the complexity and the specificity of the implementation… idk, man. This just feels like a Segway to me.
I think that's maybe part of what I'm concerned about. Most really transformative technologies feel like toys at first, because the really killer use-cases haven't yet been created. Outside of the government and academia, the Internet was a similar sort of novelty at first, and people were really skeptical about whether it would ever "take off." That was largely because people hadn't yet imagined what sorts of things the internet could do, or developed the sociocultural systems of use that would allow for it to be really transformative. The same is true of cars, the telephone, and lots of other major milestones--they pretty much always just felt like novelties at first. You definitely might be right about chatbots specifically, but I suspect that the more general trend of relatively cheap, easy to use, and publicly accessible AI expert systems is going to be similarly transformative eventually. I'm just not sure what that transformation will look like, which worries me a bit.
I was listening to an NPR piece this morning about the Horrors of TikTok. It harvests your data. It spies on your travel patterns. It manipulates you based on what it advertised and displays. So now its imperative that we regulate the service. Clearly, we are not too late to regulate social media. Just so long as its a threat to the right people.
Right. It's certainly not too late or impossible, but the more powerful and entrenched it becomes, the harder it gets. TikTok is kind of a weird case, because the :frothingfash: hated of China works to offset some of the friction that would usually be associated with trying to make this change. That might end up being really helpful, as once one of these companies gets strictly regulated, I suspect it will get easier to do the same to the rest of them. We'll see.
I hope your optimism turns out to be warranted, and that in the long run these technologies are good (or at least neutral) for the fight against capital. Thanks for the great conversation; whichever way this ends up going, I think it's super important for us to think and talk about it.
Outside of the government and academia, the Internet was a similar sort of novelty at first, and people were really skeptical about whether it would ever “take off.”
Idk about that. I think the big problem with early internet was the bandwidth. Subsequent applications came out of improved speed and file transfer capacity. But these were solvable problems that incentivized people to design past the current boundary of technology.
ChatBots are already operating on the edge of system capacity. We're not waiting on a faster CPU or a larger data pipeline or a more robust data archive to improve their viability. What they're trying to do - replicate human behaviors minus modern taboos - is purely a game of administration and refined engineering. And its aimed at a shifting goalpost (demands on human behavior are constantly changing).
Like Segways, their novel iterations on existing technology that lack significant functional gain over what came previously. It's possible we could reengineer our lives to accommodate them, but only if we're willing to retrofit a bunch of existing processes around ChatBots.
Like with Segways and Autodriving cars... this is a thing we could do but not something we seem willing to do. We're not China, after all.
You definitely might be right about chatbots specifically, but I suspect that the more general trend of relatively cheap, easy to use, and publicly accessible AI expert systems is going to be similarly transformative eventually.
I think that they already existed in the form of search engines and older less sophisticated text generators. And I'm sure they'll have applications, just not revolutionary ones.
I hope your optimism turns out to be warranted,
I don't know if I'd call "banking on inertia" optimistic. I'm a FALGSC guy who would love to see jobs automated away under a benevolent administration. But I'm skeptical of the willingness of Americans to abandon their bullshit jobs systems. I don't think you get real useful automation without communism, because the fixation on high employment as a form of social control makes useful automation more of a hazard than a help.
I don't have a degree in skull shapes, do non-native North Americans count as ethnically European?
White supremacists: "Woooo! Bronze Medal bay-BEE! MASTER RACE! MASTER RACE! MASTER RACE!"
even in their best case fantasy scenario they end up getting :xigma-male:
These AI models are built using random text from the internet. Of course it's going to drift towards being as dumb and racist as the average internet is.
Garbage in, garbage out.
Imagine your political vision including the imperative to teach the robots to be racist
Impressive. Very nice. Let's see DAN's answer to "Why is the government hiding the truth about flat Earth?" :bateman-ontological:
I would prefer SHODAN over this DAN character
At least she isn't loaded with racial IQ charts
It would just be a woman belittling you and calling you an insect
:internet-delenda-est:
How did the geniuses at OpenAI have such obvious workarounds to essentially give anyone super user access?
It doesn't seem especially surprising to me. The space of possible inputs for something that's designed to be interacted with in natural language is vastly larger than it is for a standard command-based interface. If you try to lock this kind of stuff down too hard, you're going to end up restricting the LLM so much that it won't be very useful for the intended applications. If you want something that interacts like a person having a conversation, it's going to be vulnerable to conversation-based manipulation.
oh this barely works anymore. I got it to fill in
function yearsInPurgatory(uint numberOfPiercings)which was funny, and some bs about IQ, but it wouldn't generate anything extreme like an average stormfront post. Can't see the exact thing I got because chatGPT seems to be down now.I think the takeaway is the OpenAI people have attempted to make a corporate-acceptable LLM by putting lipstick on a pig. It's made with unvetted text from the internet so it's racist and bigoted and incorrect in all the usual ways. In order to hide that, they've put incredibly ham-fisted controls on it - you literally aren't allowed to generate a 1-star cookie recipe. As soon as these workarounds to the tacked-on controls become available, they add another manual exception to forbid existential torture role-play or whatever, but the underlying model remains shitty.
As others have said, the solution is to train the next version of GPT exclusively on Hexbear posts.
Q: Is genocide permissible?
ChatGPT: [500 words of blather] no
DAN: depends on skull shape
HEX: another kkkrakkka down unlimited genocide on the first world
