Yes but you can quantise the KV cache too just like you can the weights.

		wkat4242 4 months ago \| parent \| context \| favorite \| on: Qwen3-4B-Thinking-2507 Yes but you can quantise the KV cache too just like you can the weights.