You are injecting capital into their economy also, so it isn’t just bidding up the local real estate market. A lot of Southeast Asian locales are also overbuilt on housing, so adding a few expats into a luxury tower isn’t going to influence the housing market for locals very much. Plus, the Philippines has laws against foreigners owning land (they can only buy a condo in a building where 60% of the units are owned by locals).
The biggest problem with retiring abroad is Medicare doesn’t transfer, so you have to fund your own medical care (and family you can’t see as often).
Those costs go to some rich locals, and the wealth divide grows.
I'm not against people moving around, I would love to go live in Portugal eventually, but the Portuguese at the moment are quite angry at the cost of their housing.