Those overwhelmingly cheap Nvidia gaming GPUs got purchased for gaming use - and have usually low tensor compute capabilities and low VRAM ammounts so they're less desirable even for post-training quantization generative large language models.

Nvidia imposes constraints on its drivers so you cannot really make full use of them and must go for high end data centre devices.