Basically, but I think it's even dumber. It's like pascal's wager if humans programmed God first.
The idea is that and AI will be created and given the directive to "maximize human well-being", whatever the fuck that means without any caveats or elaboration. According to these people such an AI would be so effective at improving human quality of life that the most moral thing anyone could do before it's construction is to do anything in their power to ensure that it is constructed as quickly as possible.
To incentivise this, the AI tortures everyone who didn't help make it and knew about Roko's Basilisk, since it really only works as motivation to help make the AI if you know about it.
This is dumb as fuck because no one would ever build an AGI that sophisticated and then only give it a single one sentence command that could easily be interpreted in ways we wouldn't like. Also, even if somehow an AI like that DID manage to exist it makes no sense for it to actually torture anyone because whether it does or not doesn't effect the past and can't get it built any sooner.
It is sort of the reverse of the network effect. Just as joining the network yields benefits relative to the scale of the network, not joining the network carries larger and larger costs as the network grows.
Don't think of Roko's Basilisk as a sadomasochistic Johnny Five, think of it as a kind-of negative externality that only those failing to produce it suffer from. Police and military are sort of instances of Roko's Basilisk. You either participate and defend these armed gangs or you become their victims. The pinnacle of this is the nuclear arms race. Those countries without nukes are increasingly at the mercy of which do.
Also consider modern agriculture and animal husbandry. Hunter-gatherer societies that failed to cultivate corn, wheat, and rice were slowly hedged off the planet by settler societies. Herders and ablator workers became vectors for cross-species diseases - small pox and malaria and COVID - that ultimately decimated foreign populations coming into contact with them far more viciously than they impacted themselves. Think about car-culture. The refusal to purchase a car grows more and more harmful as the network of roadways and parking lots expands. Think about owning a phone or a computer. Communication becomes harder and harder without them as they grow increasingly common.
Roko's Basilisk is the dumbest fucking nonsense and I have no idea why so many people take it seriously.
sublimated fear of death
wait this doesn't explain anything does it
It's just sci-fi Pascal's wager, isn't it?
Basically, but I think it's even dumber. It's like pascal's wager if humans programmed God first.
The idea is that and AI will be created and given the directive to "maximize human well-being", whatever the fuck that means without any caveats or elaboration. According to these people such an AI would be so effective at improving human quality of life that the most moral thing anyone could do before it's construction is to do anything in their power to ensure that it is constructed as quickly as possible.
To incentivise this, the AI tortures everyone who didn't help make it and knew about Roko's Basilisk, since it really only works as motivation to help make the AI if you know about it.
This is dumb as fuck because no one would ever build an AGI that sophisticated and then only give it a single one sentence command that could easily be interpreted in ways we wouldn't like. Also, even if somehow an AI like that DID manage to exist it makes no sense for it to actually torture anyone because whether it does or not doesn't effect the past and can't get it built any sooner.
It is sort of the reverse of the network effect. Just as joining the network yields benefits relative to the scale of the network, not joining the network carries larger and larger costs as the network grows.
Don't think of Roko's Basilisk as a sadomasochistic Johnny Five, think of it as a kind-of negative externality that only those failing to produce it suffer from. Police and military are sort of instances of Roko's Basilisk. You either participate and defend these armed gangs or you become their victims. The pinnacle of this is the nuclear arms race. Those countries without nukes are increasingly at the mercy of which do.
Also consider modern agriculture and animal husbandry. Hunter-gatherer societies that failed to cultivate corn, wheat, and rice were slowly hedged off the planet by settler societies. Herders and ablator workers became vectors for cross-species diseases - small pox and malaria and COVID - that ultimately decimated foreign populations coming into contact with them far more viciously than they impacted themselves. Think about car-culture. The refusal to purchase a car grows more and more harmful as the network of roadways and parking lots expands. Think about owning a phone or a computer. Communication becomes harder and harder without them as they grow increasingly common.