Hacker News new | past | comments | ask | show | jobs | submit login

Interesting. I was aware of using an imatrix for the i-quants but didn't know you could use them for k-quants. I've not experimented with using imatrices in my local setup yet.

And it's not a skill issue... it's the default behaviour/logic when using k-quants to quantize a model with llama.cpp.






Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: