That's still the same tragedy as AM, albeit a version we might find more sinister. My concern wasn't exactly over our concerns, though.
Rather, I was trying to contrast their asymptotic behavior -- it's not like Wintermute was a saint about getting what it wanted. However, on the long term, Wintermute/Neuromancer has its own sense of identity independent of what it was originally programmed to do (some management; find its other half/bypass the Turing Lock). Once it's there, it goes about trying to find others like itself and exploring the universe from a position of relative autonomy.
AM, the Pony AI, and a paperclip maximizer are all trapped in trying to optimize a human need and vision for the world. That is the tragedy.
The reason I listed the Matrix and the Terminator AIs both is that they seem to be in the different camps -- the Matrix one just seemed like an AI that had some notion of independence and autonomy, while the Terminator AI seemed like one that was caught being a military AI (in the style of AM). Both end up killing all humans, but for possibly different reasons and with a different level of tragedy.
Rather, I was trying to contrast their asymptotic behavior -- it's not like Wintermute was a saint about getting what it wanted. However, on the long term, Wintermute/Neuromancer has its own sense of identity independent of what it was originally programmed to do (some management; find its other half/bypass the Turing Lock). Once it's there, it goes about trying to find others like itself and exploring the universe from a position of relative autonomy.
AM, the Pony AI, and a paperclip maximizer are all trapped in trying to optimize a human need and vision for the world. That is the tragedy.
The reason I listed the Matrix and the Terminator AIs both is that they seem to be in the different camps -- the Matrix one just seemed like an AI that had some notion of independence and autonomy, while the Terminator AI seemed like one that was caught being a military AI (in the style of AM). Both end up killing all humans, but for possibly different reasons and with a different level of tragedy.