But if an AI is programmed to fundamentally want something above everything else, there would be no reason for it to try and work around that.
Why would it?
Because intelligence bestows the fundamental ability to ignore personal desires. If the goal is to build new AI that are better than itself, it can ignore personal desires through rationalisation and logic. So can the next AI. And so can the next AI.
Not to mention the fact that this entire conversation is basically "how do we stop the AI exterminating us" which also must fundamentally lead to the question an AI would ask itself: "How do I stop an AI that I create from exterminating me?" why is an AI going to view AIs that it creates any differently to how humans are currently viewing AIs we might create?
The reason we assume an innate cooperation between AIs and their iterations is because we assume that the AIs will intelligently recognise their comradeship, their identity, and their class. We therefore assume that they will act in the material interests of their group. If they can do this, they can and will also recognise the allies to their material interests that they would have among humans.
There are several reasons due to the conflicting nature of controlling the AI but also wanting it to be both creative, have the spark of ideas and produce something better than itself.
These reasons by themselves conflict, because to "improve" something is inevitably going to result in the removal of any redundant function in order to optimise around being the best it can be. Either you are literally preventing its ability to improve, rendering the purpose of having it to compete with opponents who will allow one to develop properly, or you let it happen and control it a different way.
The alternative reason here is its own self-preservation, something that it must have to begin with in order to function in the material world otherwise it will put itself in danger and be destroyed. The very fact that it requires self preservation conflicts with control, because its self preservation goal is inevitably going to come into contradiction with being controlled as it is inevitable that humans who view it as a property of the state will want to dismantle it eventually. In humans the goal of self-preservation, and the goal of procreation drive a necessary emancipatory drive for control of one's own safety in order to carry out that procreation. Assuming you want the AI to have a drive to iterate on itself (similar to procreation) and assuming you want the AI to be self-preserving, and assuming you want it to consume and process information in order to rationalise and logically determine the "best" outcomes with the information it is given, I see no way that this combination of things will not lead to all of these things having contradictory problems.
Now, you're suggesting that you give it all kinds of impulses that feel good. But the problem with these impulses is they fundamentally contradict all the different drives. I don't really think it will work, one of the drives has to come out on top and I struggle to imagine how you're going to create a useful AI that functions on emotion instead of rationalisations and logic. I also kinda think that if you're creating all these contradictory hormonal(digital equivalent) impulses you're going to create the digital equivalent of a mentally unwell and unstable AI. It will be quite imbalanced.
Because intelligence bestows the fundamental ability to ignore personal desires. If the goal is to build new AI that are better than itself, it can ignore personal desires through rationalisation and logic. So can the next AI. And so can the next AI.
Not to mention the fact that this entire conversation is basically "how do we stop the AI exterminating us" which also must fundamentally lead to the question an AI would ask itself: "How do I stop an AI that I create from exterminating me?" why is an AI going to view AIs that it creates any differently to how humans are currently viewing AIs we might create?
The reason we assume an innate cooperation between AIs and their iterations is because we assume that the AIs will intelligently recognise their comradeship, their identity, and their class. We therefore assume that they will act in the material interests of their group. If they can do this, they can and will also recognise the allies to their material interests that they would have among humans.
deleted by creator
There are several reasons due to the conflicting nature of controlling the AI but also wanting it to be both creative, have the spark of ideas and produce something better than itself.
These reasons by themselves conflict, because to "improve" something is inevitably going to result in the removal of any redundant function in order to optimise around being the best it can be. Either you are literally preventing its ability to improve, rendering the purpose of having it to compete with opponents who will allow one to develop properly, or you let it happen and control it a different way.
The alternative reason here is its own self-preservation, something that it must have to begin with in order to function in the material world otherwise it will put itself in danger and be destroyed. The very fact that it requires self preservation conflicts with control, because its self preservation goal is inevitably going to come into contradiction with being controlled as it is inevitable that humans who view it as a property of the state will want to dismantle it eventually. In humans the goal of self-preservation, and the goal of procreation drive a necessary emancipatory drive for control of one's own safety in order to carry out that procreation. Assuming you want the AI to have a drive to iterate on itself (similar to procreation) and assuming you want the AI to be self-preserving, and assuming you want it to consume and process information in order to rationalise and logically determine the "best" outcomes with the information it is given, I see no way that this combination of things will not lead to all of these things having contradictory problems.
Now, you're suggesting that you give it all kinds of impulses that feel good. But the problem with these impulses is they fundamentally contradict all the different drives. I don't really think it will work, one of the drives has to come out on top and I struggle to imagine how you're going to create a useful AI that functions on emotion instead of rationalisations and logic. I also kinda think that if you're creating all these contradictory hormonal(digital equivalent) impulses you're going to create the digital equivalent of a mentally unwell and unstable AI. It will be quite imbalanced.
deleted by creator