>99% of the code in this PR [for llama.cpp] is written by DeekSeek-R1
Yes, but:
"For the qX_K it's more complicated, I would say most of the time I need to re-prompt it 4 to 8 more times.
The most difficult was q6_K, the code never works until I ask it to only optimize one specific part, while leaving the rest intact (so it does not mess up everything)" [0]
And also there:
"You must start your code with #elif defined(__wasm_simd128__)
To think about it, you need to take into account both the refenrence code from ARM NEON and AVX implementation."
Interesting that both de-novo and porting seems to have worked.
I do not understand why GGML is written this way, though. So much duplication, one variant per instruction set. Our Gemma.cpp only requires a single backend written using Highway's portable intrinsics, and last I checked for decode on SKX+Zen4, is also faster.
Yes, but:
"For the qX_K it's more complicated, I would say most of the time I need to re-prompt it 4 to 8 more times.
The most difficult was q6_K, the code never works until I ask it to only optimize one specific part, while leaving the rest intact (so it does not mess up everything)" [0]
And also there:
"You must start your code with #elif defined(__wasm_simd128__)
To think about it, you need to take into account both the refenrence code from ARM NEON and AVX implementation."
[0] https://gist.github.com/ngxson/307140d24d80748bd683b396ba13b...