It doesn't "greatly" affect it at all. The extra traffic and time required between curve25519 and ML-KEM768+X25519 is actually less than the jump from RSA2048 to RSA4096. Imagine how silly a person would appear if they had been this alarmist about RSA4096. When building for scales where it may eventually add up you should already be taking such scale into consideration.