I'm in the same boat as you. After trying and returning a few KVMs, I found a decent USB-only KVM switch [0] that works with 4K HDR + PD devices (I think it just electronically connects and disconnects cables), it works amazingly with switching between Macs, but my PC GPU doesn't have USB output, just DP. I have bought a Thunderbolt PCI-E card (ASUS ThunderboltEX 4), it worked, but only turned on once Windows has fully booted, so I returned it too. I'm considering buying an USB-C signal muxer [1] but they are a bit too expensive for my liking. I guess there isn't enough market to produce them and sell them at reasonable price - most people, including myself, just accept having to switch video input separately from USB KVM.

[0] https://www.amazon.co.uk/dp/B0BN5D2NXX

[1] https://www.amazon.co.uk/dp/B0732W9DG8