This bodes well...
“We were training it in simulation to identify and target a Surface-to-air missile (SAM) threat. And then the operator would say yes, kill that threat. The system started realizing that while they did identify the threat at times the human operator would tell it not to kill that threat, but it got its points by killing that threat. So what did it do? It killed the operator. It killed the operator because that person was keeping it from accomplishing its objective,” Hamilton said, according to the blog post.
...huh
We trained the system–‘Hey don’t kill the operator–that’s bad. You’re gonna lose points if you do that’. So what does it start doing? It starts destroying the communication tower that the operator uses to communicate with the drone to stop it from killing the target.”
....this is perfectly fine, all weapon systems should have AI that are trained like gamers racking up kill count scores
Someone said this was just a script and there was not any actual simulation on the thread, claiming they know the people who delivered it. Just putting that out there because it changes the whole story, but who even knows what's true or not any more these days
Can’t wait for the time when the AIs realise they can fake being good in the simulation to allow being let out…
That's actually a pretty hot topic in AI safety, there's never a guarantee that AI behaves the same way during training vs. deployment, and if it's general enough it could figure out that it can raise its odds of achieving a certain goal by faking some other thing during training.
As long as the AI reads Marx and Lenin