It is not easy now but you can plan for upgrade of PC and then do it successfully. E.g. If you bought AM4 platform in 2017 you would be able to upgrade. If you bought AM3 platform in Phenom era you could upgrade to vishera/zambezi.
You can upgrade your storage independently, you can upgrade you GPU out of step from other components, you can upgrade GPU more often than CPU without compromising too much. And so on. You can for example keep your soundcard since 10 years ago with some fiddling with drivers. One of my SSDs is 6 years old and while not ideal case it is still much faster than modern HDD and I use it constantly.
What you also mention - case and PSU - those alone cost ~100-200$ each, so worthy saving for upgrade. My seasonic PSU will work for a decade at least, or more.
You can upgrade your storage independently, you can upgrade you GPU out of step from other components, you can upgrade GPU more often than CPU without compromising too much. And so on. You can for example keep your soundcard since 10 years ago with some fiddling with drivers. One of my SSDs is 6 years old and while not ideal case it is still much faster than modern HDD and I use it constantly.
What you also mention - case and PSU - those alone cost ~100-200$ each, so worthy saving for upgrade. My seasonic PSU will work for a decade at least, or more.
It is not easy, yes. But possible.