I think you both make valid points, but I also get the sense that the article is articulating insights gained from pure math explorations into the theoretical limitations of learning, which in the article can sound "turboencabulator-speak" when compressed into words.

Maybe I should have just linked to the research paper:

[B'MOJO: Hybrid state space realizations of foundation models with eidetic and fading memory](https://www.arxiv.org/abs/2407.06324)