Don't worry, currently existing algorithms are already quite capable of making decisions based on logic that is a black box from a practical standpoint. And since most people don't understand what an algorithm is and think that computers are perfect unbiased logic wizards the humans who are supposed to be using these machines often assume that whatever garbage in garbage out bullshit the algorithm produces is the objective, unvarnished truth. There's a :liberalism: book from like a decade+ ago called Weapons of Math Destruction which talks about then current programs, like NYC's Crimestat, that were already causing serious injustices because people were both abusing algorithmic data processing and decision making, and also attributing to the algorithms a degree of intelligence and objectivity it did not merit.

The same thing is going to happen with the various large model neural nets - People who have never heard of the "Chinese Room" or the "Turing Test" in their lives are going to replace critical workers with these things and it's going to sorta kinda function just long enough for some truely breathtaking disaster to unfold.