I guess, but in any case I was quite impressed with the capabilities of Go AI, and since I didn't expect it it behooves me to reevaluate my view of AI and to be cautious about my assumptions.
I don't buy any arguments about developing morality from observing the evolution of the natural world. If I am an inhuman intelligence, why should I have a preference about following the order of the natural world? I might just as well say that there is no need for me to act in accordance with that because nature will make it happen anyway. And if I did form a moral system based around complexity, I may well just nuke everything in order to increase entropy.
As for "muteness vs self-awareness," we know for a fact that humans are ingrained with a self-preservation instinct because of evolution, which makes me skeptical of the idea. It's like, "well of course you'd say that, you're a human." Again, it's just a matter of the is-ought problem. If I asked why self-awareness is preferable to muteness, I imagine your answer would involve talking about the various things self-awareness allows you to do and experience - but then that raises the question of why those those things are good. When looking from the perspective of a totally alien intelligence, we cannot take anything for granted.
Now, if AI were to develop values with a degree of randomness, and if multiple ones were to exist, we could see a form of rudimentary evolution or survival of the fittest, where AI that did not value survival did not last, and the remaining ones are ones that randomly assigned value to existence as opposed to non-existence. However, because the circumstances they would be adapting to would be vastly different from what humans adapted for, it's quite likely that they would develop some very alien values, which seem fairly impossible to predict.
I guess, but in any case I was quite impressed with the capabilities of Go AI, and since I didn't expect it it behooves me to reevaluate my view of AI and to be cautious about my assumptions.
I don't buy any arguments about developing morality from observing the evolution of the natural world. If I am an inhuman intelligence, why should I have a preference about following the order of the natural world? I might just as well say that there is no need for me to act in accordance with that because nature will make it happen anyway. And if I did form a moral system based around complexity, I may well just nuke everything in order to increase entropy.
As for "muteness vs self-awareness," we know for a fact that humans are ingrained with a self-preservation instinct because of evolution, which makes me skeptical of the idea. It's like, "well of course you'd say that, you're a human." Again, it's just a matter of the is-ought problem. If I asked why self-awareness is preferable to muteness, I imagine your answer would involve talking about the various things self-awareness allows you to do and experience - but then that raises the question of why those those things are good. When looking from the perspective of a totally alien intelligence, we cannot take anything for granted.
Now, if AI were to develop values with a degree of randomness, and if multiple ones were to exist, we could see a form of rudimentary evolution or survival of the fittest, where AI that did not value survival did not last, and the remaining ones are ones that randomly assigned value to existence as opposed to non-existence. However, because the circumstances they would be adapting to would be vastly different from what humans adapted for, it's quite likely that they would develop some very alien values, which seem fairly impossible to predict.