That was basically the debate that occurred, the risk of not writing them down was also very large for the reasons you mention. It’s only a problem if that list is then viewed as a) what defines those rights and / or b) is an exhaustive list. Unfortunately, that is what tends to happen over time, hence the need for constant pushback on people who attempt such shenanigans.
As far as your other question, it comes back to coercion vs those rights. If some jackass wants to believe that, I can’t change those beliefs. But I can certainly support the right of self-defense or shared defense of the intended victim if that ideology is used to attack them.
It's a complex issue for sure. There was a time where I was more drawn to being against the list.
But... when you look at how the world turns, and that Jim Crow was legal well into my parents lifetime, and that some individuals fight against things like privacy rights in pursuit of their own agenda and others fight for universal payments under the guise of "general welfare", the scope provided by a written list seems like the pragmatic choice.
As far as your other question, it comes back to coercion vs those rights. If some jackass wants to believe that, I can’t change those beliefs. But I can certainly support the right of self-defense or shared defense of the intended victim if that ideology is used to attack them.