huh, never heard of this thought experiment, interesting:
First described by Bostrom (2003), a paperclip maximizer is an artificial general intelligence (AGI) whose goal is to maximize the number of paperclips in its collection. If it has been constructed with a roughly human level of general intelligence, the AGI might collect paperclips, earn money to buy paperclips, or begin to manufacture paperclips.
Most importantly, however, it would undergo an intelligence explosion: It would work to improve its own intelligence, where "intelligence" is understood in the sense of optimization power, the ability to maximize a reward/utility function—in this case, the number of paperclips. The AGI would improve its intelligence, not because it values more intelligence in its own right, but because more intelligence would help it achieve its goal of accumulating paperclips. Having increased its intelligence, it would produce more paperclips, and also use its enhanced abilities to further self-improve. Continuing this process, it would undergo an intelligence explosion and reach far-above-human levels.
It would innovate better and better techniques to maximize the number of paperclips. At some point, it might transform "first all of earth and then increasing portions of space into paperclip manufacturing facilities".
The implications really need to be said more often - people who've heard of paperclip maximizers lean libertarian. And it's one of those threads you can pull on that turns them into market socialists lol.
Yeah, I think if anything, libertarians are more easily radicalized then libs, there's already a large overlap with anarchism, and you just need to convince them that rational behavior within capitalism is often destructive to the greater whole of society.
huh, never heard of this thought experiment, interesting:
(And it need not be said, the implications are pretty clear when you replace "paperclips" with "capital")
The implications really need to be said more often - people who've heard of paperclip maximizers lean libertarian. And it's one of those threads you can pull on that turns them into market socialists lol.
Yeah, I think if anything, libertarians are more easily radicalized then libs, there's already a large overlap with anarchism, and you just need to convince them that rational behavior within capitalism is often destructive to the greater whole of society.