Actually you're 100% correct.

Feels weird to me that encoding is part of MIME, but language isn't, although I understand why.

Yeah. The reason is that charset is a specific to text types. Language can apply to many media.

Though FWIW, I think the Content-Encoding header is basically a mistake, should should been Content-Transform.