It is political choices, but it's not primarily on a city level, or a year-to-year level. The problems in San Francisco are generations in the making and often national in cause. You might read Loewen's "Sundown Towns" for a good look at a key factor to understanding American cities.

The "mentality" here is one appropriate for the person and the circumstance. As you say, his neighbors weren't aliens, they were people. Compassion and understanding were worthwhile efforts. And they remain so.