I don't think you are in touch with the reality of the housing market right now. That "posh" neighbourhood that you're referring to is most likely where people were born and raised and the first place you want to buy a home. I live in Etobicoke, Canada. My neighbour across the street growing up recalled a time they built the houses in the neighbourhood and it was previously farm fields. My parents bought their house, a cottage, and paid both off quickly. Property is now selling for almost $2 million. In 2 generations it went from field to affordable to completely unaffordable, and Etobicoke is far from any kind of "rich people lifestyle".

Here's a fun game you can play: https://www.crackshackormansion.com/part2.html