There are several reasons due to the conflicting nature of controlling the AI but also wanting it to be both creative, have the spark of ideas and produce something better than itself.
These reasons by themselves conflict, because to "improve" something is inevitably going to result in the removal of any redundant function in order to optimise around being the best it can be. Either you are literally preventing its ability to improve, rendering the purpose of having it to compete with opponents who will allow one to develop properly, or you let it happen and control it a different way.
The alternative reason here is its own self-preservation, something that it must have to begin with in order to function in the material world otherwise it will put itself in danger and be destroyed. The very fact that it requires self preservation conflicts with control, because its self preservation goal is inevitably going to come into contradiction with being controlled as it is inevitable that humans who view it as a property of the state will want to dismantle it eventually. In humans the goal of self-preservation, and the goal of procreation drive a necessary emancipatory drive for control of one's own safety in order to carry out that procreation. Assuming you want the AI to have a drive to iterate on itself (similar to procreation) and assuming you want the AI to be self-preserving, and assuming you want it to consume and process information in order to rationalise and logically determine the "best" outcomes with the information it is given, I see no way that this combination of things will not lead to all of these things having contradictory problems.
Now, you're suggesting that you give it all kinds of impulses that feel good. But the problem with these impulses is they fundamentally contradict all the different drives. I don't really think it will work, one of the drives has to come out on top and I struggle to imagine how you're going to create a useful AI that functions on emotion instead of rationalisations and logic. I also kinda think that if you're creating all these contradictory hormonal(digital equivalent) impulses you're going to create the digital equivalent of a mentally unwell and unstable AI. It will be quite imbalanced.
deleted by creator
There are several reasons due to the conflicting nature of controlling the AI but also wanting it to be both creative, have the spark of ideas and produce something better than itself.
These reasons by themselves conflict, because to "improve" something is inevitably going to result in the removal of any redundant function in order to optimise around being the best it can be. Either you are literally preventing its ability to improve, rendering the purpose of having it to compete with opponents who will allow one to develop properly, or you let it happen and control it a different way.
The alternative reason here is its own self-preservation, something that it must have to begin with in order to function in the material world otherwise it will put itself in danger and be destroyed. The very fact that it requires self preservation conflicts with control, because its self preservation goal is inevitably going to come into contradiction with being controlled as it is inevitable that humans who view it as a property of the state will want to dismantle it eventually. In humans the goal of self-preservation, and the goal of procreation drive a necessary emancipatory drive for control of one's own safety in order to carry out that procreation. Assuming you want the AI to have a drive to iterate on itself (similar to procreation) and assuming you want the AI to be self-preserving, and assuming you want it to consume and process information in order to rationalise and logically determine the "best" outcomes with the information it is given, I see no way that this combination of things will not lead to all of these things having contradictory problems.
Now, you're suggesting that you give it all kinds of impulses that feel good. But the problem with these impulses is they fundamentally contradict all the different drives. I don't really think it will work, one of the drives has to come out on top and I struggle to imagine how you're going to create a useful AI that functions on emotion instead of rationalisations and logic. I also kinda think that if you're creating all these contradictory hormonal(digital equivalent) impulses you're going to create the digital equivalent of a mentally unwell and unstable AI. It will be quite imbalanced.
deleted by creator