Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

That's at BF16, so it should fit fairly well on 24GB GPUs after quantization to Q4, I'd think. (Much like the other 30B-A3B models in the family.)

I'm pretty happy about that - I was worried it'd be another 200B+.



So like, 1x32GB is all you need for quite a while? Scrolling through the Web makes me feel like I'm out unless I have minimum 128GB of VRAM.


are there any that would run on 16GB Apple M1?


Not quite. The smallest Qwen3 A3B quants are ~12gb and use more like ~14gb depending on your context settings. You'll thrash the SSD pretty hard swapping it on a 16gb machine.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: