The irreversible global catastrophe is imperialist militaries massacring people with little risk to themselves.
The irreversible global catastrophe is assembly line robots at the Toyota plant becoming sentient and going on a freaking epic terminator style mass shooting and enslave everyone except me
why? so they can't make more?
The idea makes their professions/industries more valuable because it makes computers seem more powerful than they are
so if the computers were too good there would be no need to iterate? Kinda like with smartphones not letting you replace the battery because otherwise too many people would keep their phones for 5+ years?
You're overthinking it, this is more asking your barber if you need a haircut. Of course the barber says yes, they have a clear incentive to always give that answer. AI fears are the same - ask a computer person if they think AI is a Big Deal and they have a strong incentive to say yes.
I guess that makes sense. It also could cause them to think its less important though. If their expert opinion is that its BS, they'll think less of it.