Originally Posted by
Ele
As we can see from the NTM, AI possesses the ability to complete "tasks beyond those it has been trained to do." It's self-programmable, so it's able to do things that we haven't told it to do. A constraining source code, like Musk said, is like summoning a demon. Because AI can do things that we don't intend, we have to be damn sure that our pentagram is airtight. He pumped $10m into researching Friendly AI because he's aware of the very real possibility that shit can go south.
It is not doing things beyond what the source code tells it to, the AI is programmed to be able to reprogram itself, with the purpose of being better at its original purpose. It is not entirely self programmed as it only adds code, it doesn't create its entire code from scratch.
What things can the AI do that is not intended? If its updating class is coded properly, its updates will be purely beneficial for the task it was designed to do, other aspects would not be affected.