Desktops and laptops from 10 to 15 years ago are basically all 64 bit. By the time this removal happens, we'll be at 20 years of almost all that hardware being 64 bit. By the time hardware becomes "retro", you don't need the latest kernel version.
Lots of distros already dropped 32 bit kernel support and it didn't cause much fuss.
Because they're fundamentally different things? A house is a machine for providing weather protection. The difference between a modern house and an old one is pretty minor. A computer is a machine for doing calculations. The difference between a modern computer and an old one is - by more or less any metric you can think of - many orders of magnitude. Calculations per Watt, calculations per second, calculations per unit volume etc.
It's not even like we're breaking them. This is just the maintainers of the Linux kernel choosing to not spend their time maintaining compatibility with the old architectures.
You can put in parts and effort to maintain older computers, but when you can buy a brand new desktop for $120 or a few year old one for $40 and they blow it out of the water on performance and power efficiency, what's the reason to do so? (I'm asking in the context of a main computer, not something to run old games natively or be nostalgic about.)
A computer you buy today will be much more viable in 20+ years than a 20+ year old computer is right now. We were still in the extreme part of speed and density growth back than, and it will lessen every decade.
Ten- or fifteen-year-old hardware is still perfectly serviceable now for some modern applications. (The decade-long Intel monopoly drought of 5% generational improvements to CPU performance has a great deal to do with that.) So this is not as strong of an argument as the same sentence would be if it were said ten years ago.
It's a plenty strong statement because all that hardware and more is included, back to years before the drought. 64 bit support goes all the way back to some Pentium 4s, including all the dual core models.
Lots of distros already dropped 32 bit kernel support and it didn't cause much fuss.