If farmers can't profitably farm they sell the land off to other interests or go bankrupt. People are not choosing to go into farming....at all. thus a bankrupt farm will sit fallow, be bought by some golf course development, or be turned into a "nature preserve", reducing capacity. The dots are not hard to connect.
The dots connect differently than you suggest. Sure, maybe in a suburban area the farms turn to golf courses. Many small farms are scooped up by larger farms. Looks like almost 10% reduction in number of farms since '86 and the size of the large farms have increased.
This is obviously false. If nobody at all had gone into farming in the last 40 years, there would be no farmers left. Let me introduce you to my neighbors, the farmers. Hint: they don't live in San Francisco. Drive through Central Valley and guess how many 20-40 year old farmers are involved in all that food production.
There are much fewer farmers than there used to be. But simple math tells you that more food consumption divided by fewer farmers provides a huge opportunity for more profit per farmer. If you are politically- and business-savvy enough you can become enormously wealthy.