Land is finite, desirable land even more so. You can summer in France but you can pay more to do so than if you lived there year round. Call it an incentive, not a punishment, if you want.
But I do pay more. I had to purchase an entire house and pay taxes on it.
There is no shortage of houses here in farmland France or up in rural north England[1], which explains why houses are cheap enough that you can buy two of them. That’s the housing market working like it’s supposed to. Why does it need punishing or incentivising?
[1] there are towns within a few miles with hundreds of boarded up row houses that you could get for a few thousand pounds if you like. Our village in France is surrounded by fields as far as you can see in two directions. If you want to come live here, you can (and people often do) buy a small piece of that land and build a house.
More per house. There's no natural law saying taxes need to be equal per house, or that they need to be commensurate with services consumed by those living in them. Taxes (and tax breaks) are also used to incentivize behavior, which is the answer to your original question of "why".