Great question! These are predictive models that accept a text query, do some matrix math, and then return some text. At what point in that server-client relationship does this algorithm jump the rails and run amok?
> A popular nightmare scenario for AI is giving it access to tools, so it can make API calls and execute its own code and generally break free of the constraints of its initial environment. Let's do that now!
I also think that you're assuming we know a lot more about how these things work than we actually do; you seem to think nobody is going to hook these up to APIs that can actually modify the world, despite the barrier to doing so being incredibly low; and you don't seem to have read about the adversarial training that people have been doing between the LLMs.
It's obvious that you think everything is all safe and nothing will go wrong, and I really hope you're right. But I think it's a very dangerous assumption.
You've moved the goalposts from, "we should assume that predictive text models are sentient," to, "we should be wary of programs explicitly designed to interact unpredictably with other computer systems," something that everyone from the Amish to the NSA to Cozy Bear would agree with.
> Considering that we don’t know how the brain works so well, and we don’t understand why LLMs work so well, simply on the basis of their output I think the safest assumption is that these models do indeed have agency, or at least the capability of agency.
These systems can already interact with others, it’s not moving the goalposts, it’s common knowledge. Anyone with access to the APIs can make it happen. Or are you now claiming we are just talking about one specific LLM and not LLMs generally?
Great question! These are predictive models that accept a text query, do some matrix math, and then return some text. At what point in that server-client relationship does this algorithm jump the rails and run amok?