This is actually a harmful definition, both (1,1) and (0,2) tensors can be written as a matrix but they are very different. It's like calling vector an array but vectors require vector space and arrays are just arrays. It doesn't help that std::vector is very common in CS but 'pushing back' to a mathematical vector just doesn't make any sense

This only works in finite dimensions, which for mathematicians excludes pretty much all of the interesting cases.

Finite dimension tensors is interesting both in physics(ex: mechanics, electromagnetism, general relativity) and mathematics(ex: representation theory, differential geometry). Infinite dimensions is also used in physics(quantum theory) and in mathematics (Operator Algebras, representation theory again).

I guess? Mostly the cool applications in physics and differential geometry are about tensor fields, which are more complicated than bare tensors. You could argue that they're talking about finite dimensional tensors but tensor fields are kinda a different object (at least subjectively, to me).

A tensor field is a finite dimensional object varying over space. The space of all tensor fields is infinite dimensional. An operator in QM is infinite dimensional at a single point itself, and in QFT we have fields of such operators.