The corrosive effect that things like the Facebook algorithm have had on society is a great case study for how hard this problem really is. Not for a second do I believe that Facebook is a benevolent actor, but I also don't think they set out to undermine global civil society. They trained an algorithm to optimize human behavior for engagement with / time spent on Facebook, and the way the algorithm ended up executing that optimization ended up doing a tremendous amount of harm in ways that were difficult (if not impossible) to foresee. That's the whole thing, though: you can train an AI to pursue a neutral-ish (or even good) goal, and the way it pursues that goal might be very dangerous, because AI by design doesn't think like humans do. Figuring out how to do some design harm reduction around this stuff is both a difficult and urgent problem.