It's naive to think the danger is in self-aware evil AI. AI is a tool, and can be used as a weapon.
There's a functional power difference between a knife, an assault rifle, and a nuke. Everyone owns knives--they're in your kitchen--they can't cause much trouble. Access to assault rifles has turned out to be a dangerous problem to society, and is a contentious issue as such. Nukes are obviously too dangerous for people to own willy-nilly.
The issue is where along that spectrum AI will fall into in 6mo, 1yr, 5yrs, etc.
It's troubling to think anyone, especially on a technical forum like HN, believes GPT-4 doesn't have arms or legs (anyone can hook it up to any machinery/robotics) or can't think very fast. Its training set is a good chunk of human knowledge, and it outperforms most humans already.
Even if you assume AI won't be very clever (which seems unwise given its development pace), consider just its ability to perform thoughts and actions at the speed of a computer compared to a human. There are quite a few examples of modern military engagements where a larger, weaker, and less-well-trained force overcame better-equipped opponents, like Mogadishu, Vietnam, Iraq, Afghanistan, etc.
There's a functional power difference between a knife, an assault rifle, and a nuke. Everyone owns knives--they're in your kitchen--they can't cause much trouble. Access to assault rifles has turned out to be a dangerous problem to society, and is a contentious issue as such. Nukes are obviously too dangerous for people to own willy-nilly.
The issue is where along that spectrum AI will fall into in 6mo, 1yr, 5yrs, etc.
It's troubling to think anyone, especially on a technical forum like HN, believes GPT-4 doesn't have arms or legs (anyone can hook it up to any machinery/robotics) or can't think very fast. Its training set is a good chunk of human knowledge, and it outperforms most humans already.
Even if you assume AI won't be very clever (which seems unwise given its development pace), consider just its ability to perform thoughts and actions at the speed of a computer compared to a human. There are quite a few examples of modern military engagements where a larger, weaker, and less-well-trained force overcame better-equipped opponents, like Mogadishu, Vietnam, Iraq, Afghanistan, etc.