These models aren't even trained on FP32. The original format is FP16. And quantizing to INT8/FP8 is almost lossless.

But yes, 2.5 bits per weight is pretty insane.