You can write down any set of hopefully consistent axioms, write down any set of definitions from them, and start proving theorems. The result will be mathematics. But not all mathematics is equally interesting.
People who look at asymptotic growth are interested in what happens for all, or occasionally almost all, large n. The possibility of this kind of total order is irrelevant, and therefore uninteresting to people who are interested in that. What Tao is doing is mathematics, but not mathematics of a kind that I, personally, like.
The total order on functions is not an end-goal in itself, but a step in a proof which provides useful results.
That's what mathematics is about. You work with something, then you work with something else. When you count kittens, you can't pet the numbers anymore, but the corresponding integers are still useful.
It is a theorem that any argument that can be made with nonstandard analysis (NSA), can also be made without it. The question is therefore whether NSA helps people's intuition enough to make it worthwhile.
In elementary Calculus, it really does help people's intuition. In fact it allowed us to formalize a lot of the intuitive arguments through which Calculus was originally built. In analysis, it has helped at least some people's intuition. See, for example, Robinson and Bernstein's proof of the invariant problem. However most people in analysis have found that it isn't that hard to translate the NSA version of such proofs into more familiar terminology, and they don't find the NSA version to help their intuition.
When we go as far afield as the asymptotic growth of functions, I don't see our intuition being helped much by NSA. I could be wrong - I would have been on the wrong side of the importance of oracles in cryptography on somewhat similar intuitions - but it remains my impression.
People who look at asymptotic growth are interested in what happens for all, or occasionally almost all, large n. The possibility of this kind of total order is irrelevant, and therefore uninteresting to people who are interested in that. What Tao is doing is mathematics, but not mathematics of a kind that I, personally, like.