In my experience, the largest predictor for how often you run into homeless people isn't city size, how much money the local police have, or how the residents vote, but how walkable the area is. Homeless people go where traffic is, foot traffic especially, because panhandling needs an audience.

There is a big difference between feeling uncomfortable and being in a genuinely unsafe situation, and the less you are used to seeing the homeless, the more out of touch with reality your gut feelings are.